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ABSTRACT 


A New Approach to the Analysis of Linear Models 

(May 1969) 

Fred M. Speed, B.S., St. Mary's University 
M.S., St. Mary's University 

Directed by: Dr. R. R. Hocking 

This dissertation analyzes linear models 
by writing the model in the form Y = Xu + e 
subject to e T u = £ , where the usual assump- 
tions are made about e . It is shown that 
for experimental design models, X T X is always 
diagonal and of full rank. Included are 
methods to obtain 0 T for the classical 
design models as well as arbitrary linear 
models. Other topics are regression models, 
covariance models, estimates of fixed effects 
In mixed models, and a procedure for obtaining 
expectations, variances and covariances for 
quadratic forms. 
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CHAPTER 1 . INTRODUCTION 


1.0 Preliminaries 

The theory of linear models is one of the basic 
tools used by the statistician in analyzing data. It 
has evolved from a strictly algebraic tool, which could 
be used on hand calculators, to a sophisticated matrix 
technique which is amenable to high-speed computers. 

The classical theory of linear models has as its 
foundation the works of R. A. Fisher [13]. Fisher’s 
approach is based mainly on intuitive and heuristic 
concepts. He informs his readers that his methods are 
the logical way to analyze the data. Fisher states that 
his book is to be a handbook for research workers, expe- 
cially biologists. He makes no attempt to justify his 
results on a rigorous mathematical basis. The textbooks 
[14] , [32], [34] published between 1915-1945, for the 
most part, follow this trend. They present the statis- 
tics in a "cookbook” manner; that is, if the experi- 

si 

■if 

menter follows the algebraic recipes given, he is 


NOTE — The citations on the following pages follow the 
Journal of the American Statistical Association. 



assured of obtaining the desired statistic. In 1947, 
Bartlet [2], Cochran [6], and Eisenhart [11] stressed 
the need of putting the basic concepts supporting the 
analysis of variance on a more mathematical foundation. 
Prom then until 1962, authors like Brownlee [4], 

Cramer [7], Scheffe [28], and Wilks [36] presented the 
mathematical approach to the analysis of variance. One 
disadvantage of their works, however, is that the anal- 
ysis is derived using a strictly algebraic approach, 
and while their results are mathematically valid, they 
have lost their intuitive appeal. 

About 1955, the elements of matrix theory were 
applied to the analysis of variance. Graybill [15], 

Rao [27], and Searle [31] published textbooks which use 
matrix theory as a basic tool. The result of using 
matrices is to provide not only a rigorous mathematical 
backing for the analysis of variance but also a con- 
ceptually concise theory of linear models, hence, a con- 
cise theory for the analysis of variance. Recently, 

I 

several writers [20], [22] have been advocating using 
linear algebra in studying linear models. An appre- 
ciation of this method requires a detailed knowledge of 
linear algebra, function analysis, tensor products and 
projections . 
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This dissertation provides another improvement in 
that it provides a method of analysis that makes use of 
the very simple structure of the observations . That is, 
we assume that each observation came from a population 
with a mean and a variance . If we have more than one 
population, we may know certain relations about the 
means of these populations. All hypothesis testing and 
estimation is done in terms of the population means. 

It is shown in Chapter 3 that the classical linear 
models can be considered in this manner. Using this 
approach with matrix algebra, we have a theory of linear 
models that is both mathematically concise and concep- 
tually clear from the viewpoint of statistics. 


This approach avoids the misunderstanding caused 
by imposing "nonest imable" conditions on the nonfull 
rank models. It provides an understanding of using 
additional information in mixed models to obtain better 
estimates of the fixed effects. It avoids using spe- 
cial formulae when there are pissing cells. There is 
no need to have one theory for the balanced case and 
another theory for the unbalanced case. This approach 
also suggests a technique, which is computationally 
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efficient, to find expectations, variances and covari- 
ances of Y t AY and Y T BY where Y ~ N(u,V) , and V is 
not necessarily diagonal. 

The value of this approach as a teaching tool is 
fourfold. First, the student uses fundamental concepts 
and relationships to derive the results. Second, the 
student is taught basic principles that apply both to 
the equal and unequal number per cell case. He is not 
taught a lot of "tricks" that apply only to the bal- 
anced case — "tricks" that would throw him into a 
quandary when he encounters missing cells, etc. Third, 
there is no question about what is being tested. The 
only way to obtain a sum of squares for an F test is 
to specify that the hypothesis to be tested is 

m m 

H q : X u = £ , where X is the hypothesis matrix, 

and E, is known (usually zero). Finally, the inter- 
pretation of interactions and main effects has more 
meaning since these are related to means of populations 
and their interpretation in this light is clear. 

As a computational tool, this approach is designed 
for high-speed computers. Since we have established 
the relation between this approach and the classical 
models, whenever we have a balanced experimental design 
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model, we will use the special computational methods to 
obtain estimates or sum of squares and then continue the 
analysis from our viewpoint. A computer program, which 
was written in FORTRAN V, efficiently performs the 
analysis described in this dissertation. 

The dissertation has seven chapters. This chapter 
contains the definitions and theorems necessary for 
understanding this approach. Chapter 2 provides the 
theory for analysis of the general "u" model. Chapter 3 
provides the relationships between the classical models 
and the "u" model. Chapter 4 covers the analysis of 
mixed models. Chapter 5 provides analysis for regres- 
sion models and covariance models. A discussion of 
tests for main effects and interactions also is included 
in Chapter 5 • Chapter 6 gives several numerical 
examples illustrating the theory developed. And 
Chapter 7 considers some of the research that still is 
needed in this area. 

1.1 Definitions and Theorems 

In this section, we will give the definitions and 
theorems necessary to understand this approach. The 
definitions are all referenced. The first group of 
theorems, 1.1 to 1.15, is also found in the literature 
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and has been referenced. The next set of theorems, 

1 . A to l.P, has been proved by the author and the 
proofs are included. 

Definition 1 [15] — An nxn matrix A is said to be 
idempotent if A • A = A . 

Definition 2 [15] — An nxn matrix P is said to be 
orthogonal if P T P = I . 

Definition 3 [15] - If A is an nxn matrix and X and 
a are such that AX = aX , then X is said to be an 
eigenvector of A and a the corresponding eigenvalue. 

Definition 4 [15] — When we write Y T AY, we mean that 
A is symmetric. 

We will assume that 

Y = Xu + e 

where, unless otherwise indicated, 

X is a full rank matrix, 
u is a constant vector, and 
e ~ N(0,a 2 I) . 
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We will write 


min Q(u) 

0 T u=<5 

and mean that we want to minimize Q(u) subject to 
e T u = 6 . Also 

min Q(u) 

0 T u=6 

A T u=£ 

means that we want to minimize Q(u) subject to 0 T u = 6 
and A t u = £ . 

Theorem 1.1 [ 27 ] — Let A be an arbitrary rxp matrix. 
The four equations 

(i) AXA = A 

(ii) XAX = X 

(iii) (AX) T = AX 

(iv) (XA) T = XA 

have a unique solution X . We call X the general- 
ized inverse of A and denote X by A + . 

If the vector equation Ay = b is consistent, 
then the general solution is y = A + b + [I - A + A]z 
where z is arbitrary. Note that AA + b - b and 
A[I - A + A] * 0 . 
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The vector equation Ay = b is consistent if and 

only if AA + b = b . 

If A is rxp where r < p and A is of rank 
r , then the rank of I~A + A is p-r. 

Also note that A + = (A T A) + A T 

Theorem 1.2 [27] - Let Y ~ N(u,Z) . If Z = AY + 6 Q , 
where 6 Q is fixed, then Z ~ N(Au + AEA T ) , 

Theorem 1.3 [15] — Let Y ~ N(u,a 2 I) , and let 

m k V T. ,, 

- V* V 

_ 2 / j 2 » 

c fsf a 

where n. is the rank of A. . Any one of the three 
conditions listed below is a necessary and sufficient 
condition that the following two statements be true 

(i) Y T A.Y /a 2 ~x 2 ( n -»^.) i where X. = u T A.u 

i A i * 1 1 1 x 

(ii) Y T A i Y, Y T A j Y are independent if i / j . 

1 

* 

The conditions are: 

(1) A^^ is idempotent for all i = 1 , • • • ,k 

(2) A i A j = 0 for all i 1 j 

(3) n = En i 
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Theorem 1.-4 [15] - If Y - N(u,V) 


> 


then Y T BY Is dis- 
tributed x 2 (k,X), where k = rank B and X = u T Bu , 
if and only if BY is idempotent. In the case where 
2 t Y T AY 

V = a I and Y BY = — r— , the necessary and suf- 

or 

ficient condition is that A 2 - A . 

Theorem 1.5 [27] — Let A be an mxm symmetric matrix 

with eigenvalues X. > X _>•••> X , and P *»*,P 

1 z m 1 m 

corresponding eigenvectors. Then 


x t Ax _ -i 

rji ' ^ T 

X X X 


or 


min 

x 


x t Ax 

x T x 


X 

m 


Theorem 1.6 [15] - If T ~ x 2 (Pj^) » z ~ X 2 (r) and 

rT 

T and Z are independent , then u = ^ is distributed 
as F[p,r,X] . 

I 

Theorem 1.7 [27] - Let R 2 - min [Y - Xu] T [Y - Xu] sub- 
ject to A t u - K and where X is not necessarily of 
full rank, then 

R 2 * R 2 + SS(A t ,?) 
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where 


R 2 = Y T Y - u* T (X T X)u* 

SS(A T ,£) = [A T u* - C] T [A T (X T X) + A]" 1 [A T u* - XI 
u* - (X T X) + X T Y . 

Theorem 1.8 [21] - Let Y ~ N(u,V) . Let S ± = Y T AY 
and S 2 = Y t BY . Then we have 

(i) E[S 1 ] = Trace[AV] + u T Au 

(ii) V[S r ] = 2 Trace [AV] 2 + 4u T AVAu 

(ill) Cov [S ,S 2 ] = 2 Trace [AVBV] + 4u T AVBu 

Theorem 1.9 [15] - Let Y ~ N(u s V) . Let T = AY, 

= Y T BY and S 2 = Y T FY , then 

(1) T Is independent of if and only if 

AVB = 0 

(ii) and S 2 are independent if and only if 

BVF = 0 . 


Theorem 1.10 [15] — Let A be an n*n symmetric matrix 
with rank n . There exists a nonsingular matrix P 
such that P t AP = I . 
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Theorem 1.11 [ 15 ] — Let A be an n*n symmetric matrix 
with rank n . There is an orthogonal matrix H such 
that H t AH = D where D is a diagonal matrix of 
eigenvalues . 

Theorem 1.12 [ 30] — Let Y ~ N(u,V) . Then 
Cov (Y,Y T AY) = 2VAu . 


Theorem 1.13 [27] - Let be unbiased estimates 

of 0 , and let V denote the covariance matrix of 

the t.'s. If we want to choose a T = (a a ) so 

x x n 


that 



e 


and 



a. t . 
1 1 


is a minimum, the optimum choice of a is 


a 


V'H 

£ t V -1 £ 


where = ( 1 , •••,'!) . 


11 



Theorem 1.14 [27] — Let Y = ¥6 + e where W is an 
nxp matrix of rank q < p , 3 is a constant vector 
and e ~ N(0,a 2 I) . 

then 

T 

(1) minimum variance unbiased estimate of X 3 
is A T 3 , where 3 = (W T W) + W T Y = W T Y 

2 

(2) minimum variance unbiased estimate of a is 

o 2 = CT - W3] T [Y - W3] 

= mln [Y - W3] T [Y - W3l 
3 n - q 

Theorem 1.15 [15] — Let A, B, C be n*n matrices, then 

(i) Trace(AB) - Trace(BA) 

(ii) Trace (ABC) = Trace(BCA) = Trace(CAB) 

(iii) If A is also idempotent 
rank (A) = Trace (A) . 

Theorem l.A — Let A 2 = A and B 2 = B| . If A-B is 
positive semidefinite , then AB = B . 

Proof — Let B = [b 1 ,* , »*,b ] . In order to show 

AB = B , we will show that Ab ± = b^^ for arbitrary i . 

Now suppose b = 0 , then Ab i = b.^ . Suppose b.^ ^ 0 


12 



then b*[A - B]b ± > 0 . Hence b^[A]b i > b^Bb ± . 

But B is idempotent . Hence Bb * b . Therefore 

T 1 

_ _ bTAb . 

bTAb . > b;b . . Let S - — - . Now S > 1 . But by 

11 11 b T b. 

1 1 

T 

x Ax 

Theorem 1.5, we see that max — ^ — occurs at the max- 

xx 

imum eigenvalue of A . Hence S < ^ MAX • But A is - 

idempotent. Hence ^ MAX = 1 • Therefore S = 1 . 

Hence b?Ab. = bfb . or bT[I - A]b . = 0 . Now since 

1 1 1 1 1 3 . 

A is idempotent I-A is idempotent . Hence 

bT[I - A]b i = b^[I - A] [I - A]b i . Let z ± - [I - A]b i . 

Now 0 = bT[I - A].b. = zTz. . Hence z * 0 or 

i x xx x 

Ab . = b, . Hence AB = B . 
x x 

Theorem l.B - Let R 2 = min [T - Az] t [T - Az] . Then 
" T z 

R 2 = T T [I - AA + ]T . 

Proof — Now it is well known that the z that minimizes 
[T - Az] t [T - Az] is a z such that A T Az = A T T , Now 
it is also well known that the normal equations always 
have a solution, hence by Theorem 1.1, one solution is 
z = (A T A) + A T T = A + T . Hence 

R 2 = [T - AA + T] T [T - AA + T] 

= [[I - AA + ]T] T [I - AA + ]T 

= T T [I - AA + ] [I - AA + ]T 

= T T [I - AA + ]T 
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Theorem l.C — Let R 2 = min [Y - Xu] T [Y - Xu] subject 
to A T u = £ where A T is of full row rank, then 
R 2 = [Y - Xu] t [Y - Xu] where 

u = [I - (X T X)" 1 A[A T (X T X)" 1 A]" 1 A T ](X T X)" 1 X T Y 
+ ( X T X ) ” 1 A [ A T ( X T X ) ” 1 A ] ^ 1 g . 

Proof — Let us construct the Lagrangian function 

L(u,p ) = [Y - Xu] T [Y - Xu] + 2p T [A T u - g] 

Now 

“ -2X T Y + 2(X T X)u + 2Ap 

and 

* 2(A t u - g) 

0 P 

Setting the partials equal to zero, we have 

(X T X)u = X t Y - Ap 
A T u = g 

hence 

u = (X T X)' 1 X T Y - (X T X)' 1 Ap 
Now, since A T u - g » we have 

g = A T (X T X) _1 X T Y - A T (X T X) _1 Ap 
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or 


p = -(A T (X T X) T A) _1 ? + (A T (X T X)" 1 A)" ;i A' r (X T X)" 1 X T Y . 

Hence 

u = (X T X) _ 1 X T Y + (X T X)“ 1 A[A T (X T X)" 1 A]" 1 C 
- (x T x) _ 1 a (a t (x t x) ~ 1 a) - 1 a t (x t x) ■ 1 x t y 
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Theorem l.E — Let R 2 = min [Y - Xu] T [Y - Xu] subject 
to A T u * £ , where A T is rxp of rank r , then 
T? 2 2 

^ ~ x (k,X) , where k = n - p + r and X is zero 
a 2 

if A t u * £ . 

Proof — Now R 2 is the min [Y-Xu] T [Y-Xu] subject to 
uew = {u/A t u = £} . By Theorem 1.1, all solutions to 
A T u = £ can be written as u = £ Q + Bz , where 
A T 6 0 = £ , A T B = 0 and z is arbitrary. Hence 
u) = {u/u = £ q + Bz} . Therefore 

R 2 - min [Y - X£ Q - XBz] T [Y - X£ Q - (XB)z] . 
z 

Let T = Y - X£ q and A = XB 

R 2 = min [T - Az] T [T - Az] . 
z 

This is in the form of the classic least-squares 
problem. Hence by Theorem l.B, R 2 = T T [I - AA + ]T , 
Now by Theorem 1.2, T ~ N[X(u - £ 0 ), <? 2 I] • Since 

k ' ' 

rn » 

T fT — A A It 

I-AA is idempotent and by Theorem 1.4, — L -= — — - — 

a 4 

is x 2 (k,X), where k = rank[I - AA + ] and 

X = [X(u - £ Q )] T [I - AA + ]X(u - £ q )] . 
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Now by Theorems 1.1 and 1.15, 


k = rank [I - AA + ] = Trace [I - AA + ] 

= n - Trace [AA + ] = n - rank (A) 
= n - rank(XB) = n - rank(B) 

= n - (p - r) - n - p + r . 


T 

Now, if A u = £ , then u = + Bz , hence 

u - = Bz , Therefore 

X = [XBz] T [I - (XB) (XB) + ]XBz 
« (XBz ) T [XB - XB(XB) + XB]z 
= 0 . 


Therefore If A T u = £ , then X = 0 and R 2 ~ x 2 ( k ) • 

Theorem l.F — Let Q = Q 1 + Q 2 where Q ~ x 2 (k,X) and 
~ x' 2 (H 1 ,X,) . If Q 2 “ Q - Is nonnegative, then 
Q 2 ~ X 2 (k - k 1 , X - X^) and Q 1 and Q 2 are 
Independent . 


Proof — We can assume without loss of generality that 

B]Y 


y t ay 

Q = and Q 


Y T BY 


hence 


Q = Y - T I A - 
^2 2 


where Y ~ N(u,a I) . Now A-B Is positive semidefinite 


and by Theorem 1.4, 


A 2 = A and B 2 » B 


Hence by 
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Theorem 1 . A, AB = B . Therefore let us consider 


Cl - A] 

+ [A 

- B] + B ; or 

= Y T [I • 

l 

> 
l f 

XJ 

+ Y T [A - B]Y + Y T BY . Now 

(1) 

[I - 

A] [A - B] = A- A- B + B = 0 

(11) 

[A - 

B]B = B - B = 0 

(111) 

1 

H 

L__J 

A]B = B - B - 0 


hence by Theorem 1. 

3, we have 

(1) 

y t [i - 

cr 2 

— Y - x (n - kj 

(11) 

Y T CA - 
a 2 ' 

B]Y ~x 2 < k 2 ,* 2 ) 

(111) 

y t [b]y 

_2 

0 

- X 2 (k 1 ,X 1 ) 

(iv) 

Y T [I - A]Y Y T [A - B]Y 

2 * 2 
a a 

Independent , 

Now u T u * 

x 0 + * 2 

+ X x but X Q = 

X„ = X - X 
2 

x , and 

n = n - k + k„ 
2 


, y t by 

and — — are 


k 2 ■ k - k i ' 
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CHAPTER 2. THE MODEL 
2. 0 Preliminaries 


In this chapter we will assume that we have 
sampled p univariate populations , where each popu- 
lation has a mean and a common variance. Also, let 
us suppose there may be certain restrictions known 
about the means. Now, while each population may 
have been sampled a different number of times, only 
those populations from which at least one sample was 
taken shall be included in the model. These obser- 
vations can be expressed as 


y . .. - u . . + e . , . 

ijk 13k 


where the means satisfy the relations 

eju = (X = 1 ••• r) 


where y ijk 
population: 


is the k th observation from the 1 (ij)tTi 


u is the vector of u. . in some order, 

13 * 


u„ is the mean of (1 $)th population. 
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e ijk are uncorrelated random variables such 
that E(e. jk ) « 0 , V(e. jk ) = a 2 , 

T 

0„ is the Ith restriction on the u . . ' s , and 
Z 13 3 


is known . 


Naturally, the number of subscripts is essen- 
tially unlimited; however, to simplify the notation, 
the number of subscripts will be kept to three. In 
order to better convey the concepts in this section, 
we will consider the following example. 


Suppose we had six populations indexed by two 
subscripts (ij), that is, (11, 12, 21, 22, 31, 32). 
Also, suppose we took the following number of obser- 
vations from each population; 2 from population 1, 

2 from population 2, 5 from population 3, 3 from 
population 4, 1 from population 5, and 5 from popu- 
lation 6. Let us further suppose that we know 


U 11 

U 12 

" U 21 


U 22 

U 11 

- U 12 

U 31 

+ 

U 32 
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where lu.. is the mean of the (i 3)th population. 
We can express the above as 

y . . , = u . . + e . . , 

^ 13k ij 13k 

subject to u ia - u j _ lj - u ±j ^ + u ±l j , = 0 for all 
i, j, y . i = 1,2,3; j = 1,2; k = 1, • * * n ij . 


In order to simplify the presentation, we can 
rewrite the general model in matrix notation. We 
have 

Y = Xu + e (2.0 

T 

subject to 9 u = £ 
where 

Y is the n*l vector of observations, 
u is the pxl vector of cell means, 

X is the nxp design matrix, 

e is a random variable such that E[e] = 0 , 
E[ee T ] = a 2 I , 

0 T is an rxp matrix of rank r that repre- 
sents the restrictions known about the means. 


E, is an rxl known vector. 



Let us express our example In matrix notation. 


We have 


Y = Xu + e 


subject to 0 T u = 0 


where 


Y Is an l8xi vector 
X Is an 18x6 matrix 
u is a 6xi vector 
e is an l8xi vector 
e T is a 2x6 matrix 
£ is a 2x1 vector 
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and where 
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T 

Because of the formulation of the model, X X 
will always be diagonal with diagonal elements equal 
to the number of observations from each population. 
One statistic that will frequently be encountered is 
u# =* (X T X ) _ 1 X T y which is nothing more than the 
vector of cell means. In our example, 

(X T X) = Diag [ 2 , 2 , 5 , 3 > 1 » 5 ] 

and 

(u*) = CY 11 ./2,Y 12> |2,Y 21# /5,Y 22 .y3,y 31 ./l ,Y 32 ./5] 

We shall now consider point estimation and test of 
hypothesis for the model described by Eq. (2.0). 

Note that the restriction is an essential part of 
the model. 


2.1 Point Estimation 


In this section we want to consider estimation 

of functions of the u. . . For the moment, we shall 

1 3 

consider only linear functions of the ^ ^ . In 
the classical analysis of experimental design models 
the question of estimability is raised, that is, not 


24 



all linear functions yield unbiased estimates. In 
particular, Graybill [15] gives the following defi- 
nition of estimability . 

"...A parameter (or a function of the 
parameters) is said to be linearly 
estimable if there exists a linear 
combination of the observations whose 
expected value is equal to the parameter 
(or the function of the parameters)." 

Now in the model described by Eq . (2.0) , we see 

that u_ is always estimable. One obvious estimate 

is y^j; • Hence, any linear combination of the u_ 

is estimable. A better estimate of u. . would be 

1 3 

the mean of the observations of the (i,j )th popula- 
tion, or in other words, the cell mean. Hence, one 
estimate of X T u would be X T u# = X T (X T X) _1 X T Y . 

Note that E[X T u*] « X T [X T X] _1 X T Xu = X T u . There- 
fore, for the model defined by Eq. (2.0), there is 
no need to be concerned with the question of estima- 
bility. Now we shall consider the problem of finding 
the best (minimum variance) liqear unbiased estimate 
of X T u , 

Theorem 2 . 1 — Suppose we wish to estimate X T u . The 
Best Linear Unbiased Estimate (B.L.U.E.) for X T u is 
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X^u where u is that value of u that minimizes 
[Y - Xu] t [Y - Xu] subject to 0 T u = £ . 

Proof — The proof will proceed in the following 
manner: (1) we shall find u , (2) we shall find 

the B.L.U.E. of X T u and, (3) then observe that the 

m rp ^ 

B.L.U.E. of Xu is equal to Xu. 

A 

Let us find u . In order to do this , we shall 
construct the Lagrangian function 

L(u,6 ) = [Y - Xu] T [Y - Xu] + 26 T [e T u - £] 

Now 

L L { u . » i ) = -2X T Y + 2 (X T X)u + 286 

d U 

and 


9L(u,6 ) 

—ft — 


e T u - 5 


Upon setting the partials equal to zero, we 
obtain 

■I 

(X T X)u + 06 = X T Y 


0 u = C 
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Now 


u = (X T X) _1 X T Y - (X T X)" 1 06 


But 

rn a . 

0 u = E 

hence 

6 = [e T (x T x)" 1 e]" 1 ce T (x T x)' 1 x T Y] 
- [e T (x T x)" 1 e]" 1 ? 


therefore 

u = [I - (X T X)' 1 0(0 T (X T X)" 1 0) i _1 0 T ]u* 

+ (x T x)~ 1 9(e T (x T x)~ 1 e)“ 1 5 ( 2 . 1 ) 


or 


u = u* - (X T X)’ 1 0(0 T (X T X) 1 0) _1 [0 T u* - El 

( 2 . 2 ) 


where u* = (X T X) _1 X T Y . 


Let us now find the B.L.U.E, of X T u. The problem 
can be restated as one of finding a T Y + d, such that 
V[a T Y + d] = cr 2 a T a is a minimum subject to 
E[a T Y + d] ** X T u given 0 T u = E • Let us consider 
d as d = y T $ , Now E[a T Y + d] = a T Xu + y^E = X T u 
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* 


which 


whenever 0 T u = £ or a T Xu + y T 0 T u = X T u 
means a T X + y T 0 t = X T . Hence, we want to minimize 
a 2 a T a subject to a T X + y T 0T = • The Lagrangian 

function is 


L(a, y jP) * c 2 a T a + 2cr*[X' i ' - a' A 'X - yVlp 


2 r ,T 


T rt T- 


Taking partials , we get 


3L (a .,y » P ) 
3 a 


2cx 2 a - 2cr 2 Xp 


9L(a,Y»p) 

3Y 


20 T p 


9L(a,Y >p) 

&P 


X T - a T X 


T a T 
Y 0 


Setting the partials equal to zero, we have 


Xp = a 


m /v 

0 T p = 0 

CTI/v 

X x a + 0y = X 


or X T Xp = X T a , which means 


= (X T X) _1 X T a * (X T X)^ 1 [X - 9y] 


28 



But e T p - o 


j 


hence 


0 T (X T X)~ 1 [X - 6y] = 0 

or 

[e T (X T X) _1 A - 9 T (X T X)“ 1 0 y] = 0 

or 

y = C0 T (X T X)" 1 0] -1 0 T [x' r X] -;L X 

Since a = Xp , we have 

a = X(X T X) _1 [X - 0C0 T (x' r X)" 1 0]" 1 0 T (X T X)" 1 X] 
Therefore, the B.L.U.E. of A T u is 
a T Y + d = 0 T [I - (X T X ) _1 0 ( 0 T (X T X ) -1 0)~ 1 0 T ](X T X) _1 X T Y 
+ X T (x' r X)~ 1 0(0 T (X T X)~ 1 0]" 1 C 

= X T {[I - (X T X)" 1 0[0 T (X T X)" 1 0]‘ 1 0 T ]u* 

+ (X T X) _1 6 ( 0 T (X^X ) -1 9 ) -1 £} (2.3) 

I 

But from Eq. (2.1), we see that a T Y + d Is just 
X T u. Hence, the B.L.U.E. of X T u is X T u. One imme-r 
diate consequence of the above is that if A = I , 
then the B.L.U.E. of u is u (given 0 T u = £)• 

And if there are no restrictions on the u. . (that 
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m 

is, 0 =0), then u = u* , a well-known result. 

The following theorem provides us with an unbiased 
2 

estimate of a 


Theorem 2.2 — Let a 2 = — _ 

J.J. jJ 

then E[cr 2 ] = a 2 , 


1 A m A 

— CY - Xu] T [Y - Xu] 


Proof — Let us consider 


Y - Xu = Y - X{ [I - (X T X)" 1 6(0 T (X T X) 1 0) 1 0 T ) 


Let 


(X T X) -1 X T Y] + (X T X) 1 (0 T (X T X) -1 0)'" 1 U 


A = 


I - (X T X)" 1 0(0 T (X T X)" 1 0)" 1 0 T 


then 


Y - Xu = 


Y - XA(X T X) _1 X T [Xu + e] 


- X(X T X)" 1 (0 T (X T X)“ 1 0)" 1 S 


Y - XAu - XAe - X(X T X) _1 (0 T (X T X) _1 0 )~ 1 £, 

H 

X[I - A]u + [I - XA (X T X) -1 X T ]e 
- X(X T X)" 1 (0 T (X T X)“ 1 0) _:L ? 


X(X T X) _1 0(0 T CX T X)~ 1 0)~ 1 [0 T u - £] 
+ [I - XA CX T X) ’" 1 X T ]e 
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9 


henc e 


But 


T 

0 u = 5 


Y - Xu = [I - XA(X T X)" 1 X T ]e (2.4) 

We are now going to show that I - XA(X T X) -1 X T 
is idempotent. We will first show that A is idem- 
potent. Let us set Z = (x T x) _1 e (0 T (X T X)" 1 e )“ 1 e T . 
Then, 

[(X T X)" 1 0(0 ,I, (X T X)" 1 0) -1 0' r ]C(X T X)" 1 0(0 T (X T X)" 1 0)” 1 0 T ] 

= (X T X)" 1 0(0 T (X T X)" 1 0“ 1 0 T 

Hence Z is idempotent; therefore, A = I- Z is 
idempotent. Now 

XA (X T X ) -1 X T XA (X T X ) -1 X T = XAA(X t X) -1 X t 

= XA(X T X) _1 X T 

Therefore, I - XA(X T X)“ 1 X T is idempotent and 
[I - XA(X T X)“ 1 X T ] T = I - X(X T X)“ 1 A T X T ] 

= i - x(x T x) _1 • [i - 

0(0 T (X T X) -1 0)~ 1 0 T (X T X)" 1 ]X T 
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= I — X[l - (X^T )' 1 

* e(e T (x T x)" 1 e)‘' 1 8 T (x T x)” 1 3x T 

= I - xa(x t x) _1 x t 

Let us set W * I - XACX^dV . Hence, we see that 
from the above, W is symmetric and idempotent . 

Hence , 

[Y - Xu] T [Y - Xu] = e T W T We 

* e T We (2.5) 


Now by Theorem 1.15, 

E[e T ¥e] = a 2 Trace[W] 

= o 2 Trace[l - XA(X T X) -1 X T ] 

= cr 2 [n - Trace (XA(X t X) - 1 X t )] 

= a 2 [n - TraceA] 

= c 2 [n - Traced 

| 

- Cx T x) _1 e( 0 T (x T x) _1 e)" 1 e T ]] 

= cr 2 [n - [p 

- Trace[(X T X)" 1 0(e T (X T X)“ 1 e)" 1 e ,r ]]] 
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= a 2 [n - [p - Trace[I rxr ]] 
= a 2 [n - p + r] 


Hence, 

52 = n=hr* [Y - xa - ]T[Y - x * 3 
2 

Is an unbiased estimate of o 

Up to this point, we have made no assumptions 
about the distribution of the e vector. We shall 
now assume that e ~ N(0,a I) . With this assumption, 
we are able to prove the following theorem. 

Theorem 2 . 3 — Suppose we are given 

Y = Xu + e subject to 0 T u = E, 

2 T 

where e ~ N(0,a I) and where Y, X, u, 0 , £ are 

A 

defined by equation (2.0). The estimates, u and 

'■'2 2 
a , of u and a , where 

u = [I - (X T X)" 1 0(0 T (X:FX)" 1 0)" 1 0 T ][(X T X)" 1 X T Y] 

+ (X T X) -1 0[0 T (X T X) -1 0] -1 £ 

ao ”1 /v rrt /\ 

0 = n - p +~r CY " Xu] [I - Xu] 
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have the following properties: 

(a) Unbiased 

(b) Consistent 

(c) Efficient 

(d) Sufficient 

(e) Complete 

(f) Minimum variance 

(g) u ~ Multivariate Singular Normal 

(h) P + r j a 2 ~ X 2 (n - p + r) 

(i) u and a 2 are independent 


Proof — The likelihood function is 


f (e;u,d 2 ) 


(2 it a 2 



~ Xu] T [Y 


Xu] | 

( 2 . 6 ) 


subject to 8 t u = § 


In order to establish some of the properties, we 

^ /\ o 

will show that u and a are essentially maximum 

likelihood estimates. We want the values of u and 
2 

cr that maximizes Eq. (2.6) where u must satisfy 
0 T u = £ . However, we shall maximize In [f(e,u,cr 2 )] 
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instead of f ( e ,u ,cr ) . Therefore, we want to maximize 

- § In (2 d r) - 8- In (a 2 ) - -A- [y - Xu] T [Y - Xu] 

T 2a 2 

subject to 8u = £ , 


Constructing the Lagrangian function, we get 


L(u,0 2 ,6) = In [f(e,u,0 2 )] + 6 T [ 0 T u - £] 


2 

Taking the partials of L(u,0 ,6) with respect to 
2 

u, 0 , 6 , we obtain 


|| = -y [X T Y - (X T X)u] +66 


= = 2 _ + [Y - Xu ] T [ Y - Xu] 


9L = fl T n _ r 

86 0 u 5 


Equating the partials to zero, we have 


(X T X)u - 060 2 = X T Y 


0 2 = | [Y - Xu] T [Y - Xu] 

■4 n 


Now, solving for u in the first equation we have 


u = (X T X)~ 1 X T Y + (X T X) 1 060 2 
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y 


we have 


But since 0 T u - £ 


C = 0 T (X T X)~ 1 X T Y + 0 T (X T X)" 1 060 2 


or 

6a 2 = -[0 T (X T X)~ 1 0]~ 1 [0 T u* - ?] 

where 

u* = (x T x) _ 1 x T y 

Therefore , 


u = u* - (X T X) _1 0(0 T (X T X)~ 1 0)" 1 [0 T u* - £] 

= [I - (X T X)~ 1 0(0 T (X T X)~ 1 0) -1 0 T ]u* 

+ (X T X)“ 1 0(0 T (X T X)~ 1 0)" 1 ? 

which is identical to Eq. (2.1). Hence, we know that 

u is (1) unbiased and (2) has minimum variance among 

estimates which are linear in the observations. It 

will be shown that owing to sufficiency and complete- 

/\ 

ness, we can extend the class in which u is best to 
include all unbiased estimates of uj . Hence, u and 
a are maximum likelihood estimates. They are con- 
sistent and asymptotically efficient as all the n^'s 
increase. Hence, a 2 is also consistent and effi- 
cient; and by Theorem 2.2, a is also unbiased. 
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Now let us consider the following 

[Y - Xu] T [Y - Xu] = [u - u] T (X T X) [u - u] 

+ [Y - Xu] T [Y - Xu] (2.7) 

given e T u = % . 

The proof of Eq. (2.7) is as follows 

[Y - Xu] T [Y - Xu] + (u - u) T (X T X)(u - u) 

= e T [W]e + u T (X T X)u - 2u T (X T X)u + u T (X T X)u 

But Xu = Xu + (I - W)e from Eq. (2.4). Therefore, 
we have 

(i) u T (X T X)u = u T X T Xu + 2 u T X T (I - W)e 

+ e T [I - W T ][I - W]e 

(ii) -2 u T X T Xu = -2u T X T Xu - 2u T X T (I - W)e 
Therefore 

(Y - Xu) T (Y - Xu) + (u - u) T (X T X)(u - u) 

| 

= e T [W]e + e T [I - W T ][I - W]e 

rri rp rn rp rn 

— 6 We + ee-eWe-e We 

T T 

+ e W We 
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= [Y - Xu] T [ Y - Xu] 

Thus, Eq. (2.7) is an identity. Therefore, the 
likelihood equation can now be written as 

f (e;u,a 2 ) = (2ira 2 ) 2 EXP ) i*. [ (n - p + r)a 2 

( 2a 2 

+ (u - u) T (X T X)(u - u)] 

a /\ 2 

Hence, u and a are jointly sufficient for u and 

2 A. A 2 

a . It can also be shown that u and o are com- 
plete (Ref. [15]). Since u and a 2 are complete, 
sufficient statistics, if a function can be found 
such that E[f(u,c 2 )] = g(u,a 2 ) , then f is the 
minimum variance, unbiased estimate of g(u,q 2 ). 

Hence, we see that both u and a 2 are unbiased 

minimum variance estimates in the class of unbiased 

o 

estimates of u and a 

Let us now find the distribution! of u . Prom 
Eq, (2.1), we have 

u = Au* + (x T x)" 1 e(0 T (x T x)" 1 e)" 1 c 

= a(x t x) _1 x t y + (x T x) - 1 e(e T (x T x) - 1 e)' 1 s 

(2.8) 
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Hence 


VCu] = a 2 (A ( X T X ) ~ 1 X T X (X T X ) ~ 1 A t ) 

= a 2 A(X T X)^ 1 A T 

- 0 2 a(x t x) -1 [i - e T (e T (x T x) _1 e)" 1 e T (x' r x)" 1 ] 
= a 2 A[i - e(e T (x T x)" 1 e)" 1 e ,r ](x T x) _1 

= a 2 AA(X T X )~ 1 

= 0 2 A(X T X)“ 1 

and u is normal by Theorem 1.2. Note that since A 
is idem£otent , V[u] is singular and hence u has a 
multivariate singular normal distribution. The reason 
for this is that u is constrained to lie in a p-r 
dimensional subspace of the p space. The implica- 
tions of this will be taken up when we consider 
interval estimation. 

/%2 

Let us now consider the distribution of 0 
Now by Eq. (2.5) we have (n - p + r)o 2 = e T [W]e . 
Therefore, by Theorem 1.4, 
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Hence , 

n - ? ■ + r 5 2 ~ x 2 (n - p ■+ r) 

a 

We will now show that u and a 2 are 
dent. To do this, we observe that from Eq. 
from the fact that 0 T u = %, that 

u = A(X T X) _ 1 X T [Xu + e] + (X T X)“ 1 e[e T (X T X) _1 0)“ 1 C 

= Au + (X T X)' 1 0(0 T (X T X)" 1 0~ 1 )5 + A(X T X) _1 X T e 

- u + A(X T X) _1 x’ r e (2.9) 

Thus u - u = A(X T X) -1 X T e . Now, by Theorem 1.9, 

u-u and a 2 are independent if (A(X T X)‘* 1 X T ) (W) - 0 . 

But 

A(X T X)’ 1 X T W = A(X T X) -1 X T [I - XA(X T X) -1 X T ] 

= A(X T X) _1 X T - A(X T X) _1 X T - 0 

( 2 . 10 ) 

Therefore u-u and a 2 are independent, which also 
implies that u and a 2 are independent. 


indepen- 
(2.8) and 
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2.2 Interval Estimation 


Since the distribution of a 2 is known to 

2 a 
be x (n-p+r ) , the procedures for interval estimation 

2 

of a are well known and will not be discussed here 

A 

However, since u has singular distribution, we 
shall discuss the problems associated with finding 

T 

confidence intervals for A u. For the moment, we 

T 

will assume that X is a lxp vector. We will post- 
pone the discussion of simultaneous confidence 
intervals until Section 2.4. 

Suppose we wish to place a confidence interval 
on X T u» Then, the B.L.U.E. of X T u is A T u and 
V[X T u] = a 2 A T A(X T X) _1 A T A . Note that A(X T X)" 1 A T is 
singular and there is the possibility that 
cr 2 A T (A(X T X)~ 1 A T ) A = 0 . Since (X T X) _1 is a positive 
definite matrix, a 2 A T (A(X T X)’"' L A' r )A = 0 if and only 
if A T A = 0 . Now we claim A T A =0 if and only if 

A = 0y . In other words, the only time V[X T u] = 0 

$ 

f 

is when we are estimating Au - y i 0.' l u = y » a 
known constant. The proof of this is: Suppose 

A T A = 0 , then we have by Theorem 1.1, 



X ~ [I - (A T ) + A T ]z , But (A T ) + = A T since A T 
is iderapotent. Hence 

X = [I - A T A T ]z = [I - A T ]z 

» e(e T (x T x)“ 1 e) _ 1 e' r (x T x)" 1 z 

« 0y where y = (0 T (X T X)“ 1 0)“ 1 0 T (X T X)" 1 z 


Now suppose X = 0y , then 

A T X = A T 0y = [I - 0(0 T (X T X) _1 0) _1 0 T (X T X) _1 ]0y 
- [0 - 0]y = 0 . 


Therefore, if u is a nontrivial function of the 


r i t t -i 

parameter, we know that - — — — - is distributed 

VvCx T u] 

as N(0,1). Since u - u = A (X T X) -1 X T e , we see that 
X T (u - u) = X T A(X T X) -1 X^e and from Eq. (2.10) 
X T A(X T X)~ 1 X T W = X T 0 = 0 , Hence by Theorem 1.9, 
X T (u-u) is independent of o 2 . Hence, 


yJ7. 


X T u - Xu // 

i T A(X T X ) _1 A T X / ’ 


(n - p + r )a‘ 

o 2 


is a 


t statistic 


with n-p+r degrees of freedom. The procedures for 
finding confidence intervals from this information 
are well known. 


42 



2.3 Hypothesis Testing 


In this section we will develop a procedure for 
T 

testing H q : A u = y against the two-sided alterna- 

tive H a : A u y , where A is an s*p matrix of 
rank s . We will use the likelihood ratio test. 

The following is a brief description of the procedure. 

Suppose x. ,x , • • • ,x are distributed as L(x,0 ) , 

JL XI 

where 0 is an element of a set ft . Now, if we 

want to test H : 0eo> , where w is a subset of 

o 

ft , versus H : 0e[ft - w] , we construct 

A 

max L(x, 0 ) 

rU) ~ maFTTxTTT 
0 eft 


Now we can observe that 

(1) r(x) is a function of x alone and hence a 
statistic 

(2) 0 < r(x) < 1 

% 

(3) small values of F(x) suggest rejection of 

H . 
o 
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Let us now apply this procedure to our model. Since 
Y ~ N[Xu,a 2 l] where e T u = K , we see that 

_n / • 

L(Y;u,cr 2 ) = (2-rra 2 ) 2 EXP \ [Y - Xu] T [Y - Xu] 

( 2c 

given 0 u ** £, . 

Hence we see that is the set of all u such that 

0 T u - £ or Q = {u/0 T u = . Now suppose we wish to 

test H : A T u = y , then w is the set of all u 

O 

such that A T u = y and 0 T u = K j or to = {u/A u = y 
and 0 T u - O . Therefore Q - to = {u/A T u / y and 
0 T u = ?} . And now we can write 


r(Y) 


2 

max f ( Y ; u , a ) 



max f[Y;u,a 2 ] 
0 t u=£ 
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3 


then we can 


max f (Y;u,cr ) 


r(Y) = 


B T u=5 


max f(Y;u ) 

e T u=e 


In Section 2.1, we showed that the values of u 
and a 2 that maximized f(Y;u,o 2 ) subject to 0 T u = £ 
were 


u = [i - (x T x) -1 e(0' r (x T x)" 1 e)' 1 e T ]u* 


+ [X T X]" 1 0(0 T (X T X)" 1 0)" 1 C 


— [Y - Xu] T [Y - Xu] , where 


u* - (x^x'Vy . 


Therefore 


max f(Y;u,a 2 ) = |2 t rcr 2 ) 2 EXP 

m ' * 


0 T U=? 
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Likewise it can be shown that the values of u and 


2 2 T 

c that maximize f(Y;u,cr ) subject to B u = 6 were 

u = [I - (X T X) _1 B[B T (X T X) _1 B] _1 B T ]u* 

+ (X T X)~ 1 B(B T (X T X)" 1 B)~ 1 6 

a 2 = i- [y - Xu] t [Y - Xu] 

Therefore sup f(Y;u, 0 2 ) = ( 2ir a 2 ) 2 EXP 
B T u=6 

Hence, we see that 



r(Y) = (?) 2 • 

_ 2 _ 

The statistic that we will use is r(Y) n . Let 

_2 

us denote T(Y) n as L* . Now we reject H q if 
L* > L* since L* > L* if and only if T (Y) > T(Y) 
The problem now is to determine the distribution of 
L* , Let us consider the identity 


where 


L* 



1 # + L 


L 



a 


2 


H6 



Let us note that by Theorem l.C, 


where 


min [Y - Xu] x [Y - Xu] 
uem. 


= min [Y - Xu] A [Y - Xu] 



w 2 = {u/0 u = £} 


Now is a subset of w 2 , hence by 

Theorem l.D, a is greater than or equal to o 

—2 ~2 

Now by Theorem l.E, and are x 2 (h ia A ) and 

o a 

X (h 2 ,A 2 ) where h 1 = n - p + (r + s) and A x is 


zero if 


and where h 2 =n-p+r and 


A 2 = 0 if 0 T u = £ . But wc are given that 9 T u = £ > 
hence A 2 = 0 and A 1 = 0 if A T u = y . Therefore 


47 



we have 


io 2 

2 


ncr _ no 2 , n ,—2 ~2, 

= — + — (a - a ) . Now 

a 


2-r- (a 2 - a 2 ) > 0 , hence by Theorem l.P, 


n (a 2 - a 2 ) ~ x 2 (s,X 1 ) and ~ (a 2 - a 2 ) is 


independent of a 2 

a* 


Therefore we see that 


L 


n(a 2 - a 2 ) 


~2 

no 


2 2 

is the ratio of a noncentral x and a central x 

Therefore by Theorem 1.6, F = — + - - L is dis- 

* s 

tributed as a noncentral F with s and n-p+r 
degrees of freedom and noncentrality parameter X , 
where X - 0 if H : A T u = y is true. Now L* > L* 
if and only if L > L ; and L > L if and only if 

O O 

F > F . Hence we reject H : A T u = y if F > F 

—• o ° o 1 — o 

And since we know the distribution of F , it is easy 

to find F . # 

o 
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Let us now consider a computational form for 
F . By Theorem 1.7, we see that 

(i) min [Y - Xu] T [Y - Xu] = R 2 + SS(B T ,6) 

o 

B u=S 

(ii) min [Y - Xu] T [Y - Xu] = R 2 + SS(0 T ,O 

0 T u=£ 

where 

R 2 = Y T Y - u* T (X T X)u* 
o 

and 

SS(B T , 6 ) = [B T u* - 6] T [B T (X T X)“ 1 B]“ 1 [B T u* - 6] 

and 

ss(e T ,?) = [e T u* - £] T [e T (x T x) -1 e] _1 [e T u* - ?] 

where u# = (X T X) -1 X T Y . Hence P can be written as 

p * n - p + r [SS(B T , 6 ) ] - SS(0 T ,g)] (2>11) 

s R 2 + SS(0 T ,5) 

Let us also note that 

= E[SS(B, <5 ) - SS(0 T ,S) ; j 
= [B T u - 6] T (B T (X T X)" 1 B) _1 [B T u - <5] 

- (0 T u - C) T [e T (X T X)" 1 0] _1 [0 T u - ?] 
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X 



But we are 


hence 


given 0 u - g , 

X = [b T u - 6] T [B T (X T X) _1 B]" 1 [B T u - 6] 



' 0 

-*l 

T f 0 

= 

_A t u - 

- Y_ 

[B T (X T X)" 1 B]“ 1 A T u - 6_ 


= [A T u - y] T [B T (X T X)‘ 1 B]" 1 [A T u - 6] . 

Another form for the numerator sum of squares 
will be given at the end of Section 3.0. 


2.3*1 Choice of A T for H : A T u = y . 

T 

Some care must be taken when choosing A for 
T 

H q ; A u = y . For example, suppose we are given the 
model Y = Xu + e subject to ^ - u = 2 . The test 
H q : U-j_ - u 2 - 0 is obviously meaningless. In other 

T 

words, we must choose A such that 


- • El 


is a consistent set of equations and rank 


= r + s 
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2.3.2 Example 


Let us consider the example in Section 2.0, 

T 

There we were given Y = Xu + e subject to 0 u = % , 
where 



Now suppose we wanted to test H q : u 1# ■- u = u.^ 

T 

or H : A.u = 0 where 

o 1 



Hence we have 



where n - p + r = 18 - 6 + 2 = 14 and s = 2 . 
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Let us consider another hypothesis- Suppose 

we wanted to test H : u.. = u... for all i.i'i. 

o 13 x 3 

T 

The A 2 matrix would be: 


10 -1 0 0 0 

a t 1 0 0 0 -1 0 

2 0 10-100 

0 1 0 0 0 -1 


T 

Now, when 0 is augmented to 


to form 



T 

we see that B 2 is a 6x6 matrix of rank 4, that is 



If we add the third row to minus the first row,, we 
obtain the fifth row; and minus the second row plus 
the fourth row yields the sixth row. , Therefore, in 

T 

light of Section 2.3.1, we would take B 2 to be 
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Let us note that - HB^ where 

1 0 0 o - 

H 0 10 0 
-1 0 2 0 

0-102 

We will show later that this Implies that 
H : u = u = u is equivalent to H : u. . = u 

O.L ^ J O X j 1 1 j 

for i, i’, j . This example points out the care 

needed in selecting the hypothesis and the constructing 
of the B matrix. By properly setting up the B 
matrix, the experimenter will know exactly what he is 
testing and will be guaranteed a valid test. 

2.4 Simultaneous Confidence Intervals 

In this section, we will be concerned with finding 

T 

confidence intervals for A u = 6 , The procedure is 
as follows. By Theorem 2.3, we know that 
u - u ~ N ( 0 , A(X T X)“ 1 A T ) . Hence 

A T (u - u) ~ N(0, A I? A(X T X)- 1 A T A) . 

Now, from Eq. (2.9) we see that 

A T (u - u) = A T A(X T X) -1 X T e . 
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Consider 


T 2 = i- [A T (u - u)] T [A T (A(X T X)" 1 A T )A] + [A T (u - u)] 
or 

T 2 = e T [X(X T X ) -1 A T A (A T A(X T X) -1 A T A ) + A T A(X T X) -1 A T ]e 

<J 

= e T G(G T G) + G T e 

where G = X(X T X) -1 A T A . But 

[g(g t g) + g t ]g[g t g] + g t = G(G T G) + G T , 

hence by Theorem 1.4, T 2 ~ x 2 (k) , where k = rank(G) 
Since A is not of full rank, we must consider the 
possibility that A T A = 0 . However, by following a 
similar argument as put forth in Section 2.2, we see 
that A T A =0 if and only if A T = r T 0 T or 
A T u = f T 5 , a known vector. Hence if A T u is a non- 
trival function, then A A ^ 0. Now 

2 _ T [G(G T G) + G T l3 

e 2 

a 

is independent of e T We since G(G T G) + G T W = 0 . 
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Therefore 


T 2 /k 

g 2 /a 2 


F(h, n — p + r) 


Hence 


,| [A T i 


(u - u)] T [A T A(X T X)~ 1 A T A] + [A T (u - u)3 < p L ! _ a 
kc 2 a | 


But since 6 = A u , the set of <S such that 


(A t u - 6) T [A T A(X T X)~ 1 A‘ r A]*[A T u - 6] < p 

to 2 ' 0 

is a (1-a) simultaneous interval for 6 = A u . Now 
if 0 = 0, then A =• I and. 

T 2 = ir- [A t (u* - u)] t [A t (X t X)’ 1 A] _ 1 [A t (u* - u)] 
cr 

which is the classical result. Numerical examples of 
the above procedure will be given in Chapter 6. 
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CHAPTER 3. RELATION TO OTHER MODELS 
3 . 0 Preliminaries 

We will now consider the relation between the "u" 
model discussed in Chapter 2, that is Y = Xu + e sub- 

T 

ject to 0 u = 5 , and the classical linear models. 

The assumptions for the classical models are very sim- 
ilar to the assumptions for the "u" model. For the 
classical linear models, we assume that we have sampled 
p univariate populations where each population has a 
different mean, but all have the same variance. 

The functional form for these models is Y = Xu + e 
subject to u - Pg or simply 

Y = Wg + e (3.1) 

where W = XP and P is p*t of rank q . We shall 
refer to this model as the ”g M model. 

We shall assume that e ~ N(0,a 2 I) , since it is 
this case that is most interesting. The case in which 
we assume E[e] = 0 and E[ee T ] = a 2 I has limited 
application and the needed results can be obtained by 
following a similar line of reasoning as for the case 
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2 

where e ~ N(0,a I) . In order to establish a relation 
between the "u" model and the ”B" model, we shall define 
the concept of statistical equivalence between two 
models. We say the "u" model and the "B" model are 
statistically equivalent if: 

(i) there is a 1-1 correspondence between linear 
estimable functions of the parameters, and 
minimum variance unbiased estimates are 
identical. 

(ii) there is a 1-1 correspondence between test- 
able hypotheses, and the test statistics 
(under the same criterion) are identical. 

Now suppose we are given Y - WB + e where 
W = XP , or in other words, u = PB . Let us consider 
the following "u" model: Y = Xu + e subject to 

T 

0 u = 0 where: 

(1) 9 t P = 0 

(2) rank(0 T ) = rank ( X ) - rank(XP) = p - q (3.2) 

(3) 0 T is (p-q)xp of rank p-q . 
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In light of the above, we are able to prove the 


following theorem: 

Theorem 3.0 — The "u" model as defined by Eq. (3.2) is 
statistically equivalent to the ”8" model as given by 
Eq. (3.1) . 

Proof — By Theorem 1.14, the minimum variance unbiased 
estimate of a 2 in the model is given by 

n 2 « min [¥ - we] T [Y - W0] 

li — CJ g 

where q « rank(W) . And from Theorem 2.3, the minimum 
variance unbiased estimate of a in the "u" model is 
given by 

O 2 * r min [Y - Xu] T [Y - Xu] 

o *** p * X* 

subject to 8 t u * 0 . But r * rank(0 T ) = p - q . 

Hence n-p+r«n-q . Now 0 T u = 0 implies that 
U * £1 - (0 T )*0 $ ]z where z is arbitrary. But we are 
given that i^P * 0 ; hence jP - [I - (0 T ) + 0 T ]G , 
where G is chosen so that P is p*t of rank q . 
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Let f . z, =, Gy . Hence, ,u = [I - ( e T ) + e T ]GY: = Py . There- 
fore , we have ; i-- ** 

. " = n ~ - p + r mln CY - Xu] t [Y •*' Xu] ^ 

e T u=° 

= — - min [Y - Xu] T [Y - Xu] 

n ~ q u=Py 


= p min - Wy] T CY - Wy] 

= a 2 , (3.3) 


2 

Hence the minimum variance unbiased estimate of a in 
the "u" model is identical to the minimum variance 
unbiased estimate of a in the "0" model. 

T T 

Now suppose we wish to estimate 6 3, and that 6 3 
is estimable; i.e., there exists an a such that 

»p ip 

6 = a W . Since u - P$ , we have 

3 - P + u + [I - P + P]z . 

I 
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Hence 6 T g = <S T P + u + 6 T [I - P + P]z . But 6 T = a T W or 
6 T = a T XP . Therefore, we see that 

6 t [I - P + P]z = a T XP[I - P + P]z 

= ' a T X[P - P]z = 0 

Hence 6 3 = 6 P u . Now let us suppose we want to 
estimate X T u in the "u" model. Since u = P$ , we see 
that X T u = X T Pg . Therefore, we have a 1-1 corre- 
spondence between estimable functions; that is if we 
want to estimate <S T g, then we can estimate X T u where 
X T = 5 t p + , or if we want to estimate X T u we can esti- 
mate fi T g where 6 T = X T P . 

Prom Eq. ( 3 * 3 ) > we see that 

min [Y - Xu] T tY - Xu] = min [Y - Wg] T [Y - Wg] 
0 T u*O e 

A 

Also from Eq. ( 3 • 3 ) we see that u is that value of u 
that minimizes [Y - Xu] T [Y - Xu] subject to 0 T u = 0 ; 

A 

and g is that value of g that minimizes 

I 

[Y - Wg] T [Y - Wg]. 

A A 

We will now show that u = Pg , that is we will 
show that Pg minimizes [Y - Xu] i [Y - Xu] subject to 


6l 



0 T U = 0 . Now let u = P3 . We see that 

e T u = 0 T P3 = o 

Hence u satisfies the constraints. Also 

/ 

[Y - Xu] T [Y - Xu] = [Y - XP 3 ] T [ Y - XP3] 

= min [Y - XP3l T [Y - XP3] 

3 

= min [Y - Xu] T [Y - Xu] 

subject to 0u = 0 . Hence u = u , or u = P3 . 

Now by Theorem 2.3, the minimum variance unbiased esti- 
mate of X T u is X T u = X t P 3 = 6 t 3 if 6 T = X T P . But 
by Theorem 1.14, o3 is the minimum variance unbiased 
estimate of <5 T 3. Hence if we wish to estimate 6 T 3, 

T 

then the minimum variance unbiased estimate of 6 3 in 
the "u" model is X T u where X T = 6 T P + . The converse 
also is true. That is, suppose we wish to estimate 
X T u, then the minimum variance unbiased estimate of X T u 
in the "3" model is 6 T 3 where 6 T = X T P . Thus condi- 
tion (i) is satisfied. j 

If we wish to test H : 6 T 3 = y in the "3" 

o 

model, we see that this is equivalent to testing 

H : X T u = y in the "u" model where X T P = 6 T . The 

o 1 
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reason for this is the 1-1 correspondence between esti- 
mable functions. Now, if we use the likelihood ratio 
test as a test criterion, the test statistics are iden- 
tical. This can be established by considering the 
following: 

= min [Y - Xu] T [Y - Xu] 

0 T u=O 

X T u=y 

= min [Y - Xu] T [Y - Xu] 
u=P3 

A T u=y 

= min [Y - W3] [Y - W3] 

6 T P + P3=Y 

but <S T = a T W » a^XP . Hence 

<5 T P + P3 = a T XPP + P3 = a T XP3 = S T 3 

Hence 

= min [Y - W3] T [Y - W3] 

1 S T C 

6 3=Y 
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Now the appropriate test in the "u" model is 


P 


min [Y - Xu] T [Y - Xu] - min [Y - Xu] T [Y - Xu] 
0 T u=O 0 T u=O 

,1 T u=y 

S0 2 y 


but 


min [Y - Xu] T [Y - Xu] 
0 T u=O 


min [Y - W3] T [Y - WB] 
6 T g=Y 


and 


min [Y - Xu] T [Y - Xu] 
6 T u=0 


= min [Y - Wg] T [Y - WB] 
B 


and d 2 = 


A 2 

a hence 


P = 


min [Y - 
<S T B-Y 


wb] t [y 


WB] - min [Y 
B 


- WB] [Y - WB] 


sa 


which is the appropriate test in the "3" model.- Hence 
condition (ii) is satisfied and the theorem is proved. 

The importance of Theorem 3.0 is that if we are 
given Y = WB + e where W is an n*t matrix of rank 
q < t , we can transform this to the appropriate ’’u" 



model and perform the analysis In the "u" model. Hence 
we have the ability to work In either model. 


Let us now consider another form for the numerator 
of the test statistic. From, Theorem 1.7, we see that 


min [Y - W3] T [Y - W$] 
S T $=y 



where 


and 


Hence 


R 2 = min [Y - W3] T [Y - W3] 
0 3 


V[S t 3] = a 2 6 T (W T W) + 6 


min 

6 T 3=Y 


[Y - W3] T [Y - W3] - min [Y - W3] T [Y - W3] 

3 


= (S T 3 - Y)’ r (5 T (W T W) + 6)" 1 (6 T 3 - y) 



- R 


2 

0 


Now we know, by Theorem 3.0, that 


rn /s m 

Xu = 6 T 3 
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where 


Hence 


6 T - X T P , 


V[Xu] = V[6 T e] . 


Therefore, we can write 


R l " R o = [xT ^ “ y]T 


V[X 


-n 

■f- [x T G - 

r 


y] 


but 


V[X T u] ~ a 2 X T A(X T X)~ 1 X . 


Hence 

R 2 _ r 2 = [x T u - y ] T C X T A ( X T X ) “ 1 X ] ” 1 [ X T u - y] . 

Thus the test statistic can be written as 

p = [X T u - y] T [X T A(X T X) ~ 1 X]~ 1 [X T u - y] . (3>i , 

sa 2 

Hence we have two forms for the test statistic. 

We shall now consider specific models usually 
encountered in statistics and show how they can be 
analyzed in the "u" model. One important aspect of 
using the "u" model should be noted; and that is that 
there is no need to distinguish between the so called 
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"equal numbers" case and the "unequal numbers" case. 

The theory developed In Chapter 2 is completely gen- 

T 

eral. If we indeed have equal numbers, then X X = rl 
and all of the formulas simplify. Furthermore, our 
analysis does not have two sets of equations as do some 
of the analyses of linear models. For instance, 
Graybill [15] has a section in which he analyzes the 
two-way without interaction assuming equal numbers and 
then a chapter devoted to the analysis when he has 
unequal numbers. 


3.1 Classification Models 

In this section, we will consider classification 
models, that is one-way, two-way without interaction 
and the two-way with interaction. Extension to 
higher-way classifications is "easily" seen. 


3.1.1 One-Way Classification 

Suppose we are given the usual one-way classifi- 
cation, that is 



u + a . + e . . 
i i] 


i = 1, • • • ,a,J 


1»**’ J n i 
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Now we can rewrite this in matrix notation as 

Y = ¥3 + e , where W is nx(a+l) of rank a . At 

this point, we would like to find the appropriate "u" 

T 

model. Prom Eq. (3.2), we see that we want 0 such 
that rank(0 ) = rank(X) - rank(W) . But rank(X) = a 
and rank(W) = a . Hence rank(0 T ) = 0 which implies 
0 = 0 . Therefore the appropriate "u" model is 



u . + 


e . . 
1 3 


i = 1, 


j = 1, 


.n , 


In other words the u.’s are unconstrained. Note, how- 

i 

ever, that we do know that u. = u + a. or in matrix 
notation u = P3 . Therefore let us consider hypoth- 
eses about u and in the ,, 3" model. For example, 

suppose we want to estimate o^-o^. We need to find a 
X T such that X T P = <$ T . It is clear that 


u 


l 


u. 


T 

X u = - a 2 


S t 3 


Hence to estimate a -a, in the "3" model we estimate 
u 1 ~u 2 in the "u” model. Since 0 T = 0, u 1 = Y 1# ^n 1 and 

"2 = Y 2-/ n 2 * HenCS 



which is a well-known result. Now suppose we wanted to 
test H q : <» 1 ■* a 2 *■•••• .a . The appropriate test 
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in the "u" model is H : u, - u„ = • • • = u . We 

shall present numerical examples of this test and sev- 
eral others for the one-way classification in Chapter 6. 
The important fact here is to note that given the " 3" 
model y = u + + e , the appropriate "u" model 

is y . . = u , + e . . . 

ID i 13 


3.1.2 Two-Way Classification Without Interaction 

The usual definition of the two-way classification 
without interaction model in the " 3 " model notation is 


y 


i jk 


= U 


4 a . + y ■ 4 e , . 
i '3 13 k 


i = 1, • • • ,a; j = 1, • • • ,b; k = 0,1, •••,n i . In other 

words, we assume that the model is additive. 

Graybill [15] gives the following definition of the two- 
way additive model. 


"The two-way classification model ' 

y. , = u. . + e. . will be said to be 

an additive model if u. . is such 

id 

that u.. - u . , . - u.,, + u . , . , = 0 
id i'd id' I'd* 

for all i, i', j, j' ." 


This definition provides us with the appropriate 
'u" model; that is Y = Xu + e subject to 0 T u = 0 , 
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T 

where a typical row of 0 is 


u . . 
13 



u . . . 
13 


+ u. 
x 


0 . 


Now rank(0 T ) = rank(X) = rank(XP) . The rank of X 
is, say, p and rank(XP) = a + b - 1 . Hence, by 
Theorem 3.0, rank(0 )=p-a-b+l . Now let us 
consider various possibilities. First let us suppose 
i = 1 , • • • ,a; j = l,***,b, and k = 1 . Then p = ab and 
rank(0 T ) = ab - a - b + 1 = (a - 1) (b - 1) . Also 


a 1 2 


1 

n - p + r 


[Y - Xu] T [Y - Xu] 


ab 


1 

ab + r 


[Y - a] T [Y 


u] 


“ Ta 


1 

1Mb - 1) 


[Y - u] T [Y 




Hence the degrees of freedom for residual are (a-l)(b-l) 
as is well known for this case. 


Second, let us suppose i = 1, 


and k = 0 , 1 , • • • ,n 


13 


In this case. 


which is the number of nonzero n. . . 

13 

of freedom for residual are n -a-bd-1 

V'' V* 


n, 

matrix. 


= 4 . 2 . n .. 

1 0 13 


In this case , 0 ' 


•»a, J = 1 , • • * ,b 
rank(X) = p , 

Hence the degrees 
, where 

is a (p-a-b+l)xp 
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for the 


T 

Let us now consider the structure of 0 

above two cases. First, let us consider the following 

example: Let i = 1,2, 3,4, j = 1,2,3, and k = 1 . 

T 

Since a method of constructing 0 is given in 
Chapter 5, we will not now discuss the details con- 
cerning the construction of 0 . We will only observe 

the structure of 0 T . The constraints on the u^ 
were that u. . - u. , . - u. . . + u. , . , =0 for all i, j 
i 1 , j ’ . Hence we have 



0 

0 

0 

0 

0 

0 


Note that we do not have the constraints of all 
ij i 1 , j, j 1 . This is because the above six form a 
linearly independent set , and any other constraint 
would be a linear combination of these six and need not 
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T 

be included in 0 , This is in keeping with the 

T 

assumption that 0 be of full-row rank. In matrix 
notation, we have 



Let us now consider estimates and tests of hypoth- 
esis for this example. The "3" model is 

y . . = u + a . + y . + e , . 

13 1 3 13 

The appropriate "u" model is 

Y = u + e 

T T 

subject to 0 u = 0 where 0 is given above. Now we 
know that 

u . . = u + a . + Y • 

13 13 

Suppose we wish to estimate a i “ a i > then 

u..-u.,.=a. -a., : or, if we wish to estimate 

13 1 ' 3 x 1 ' 
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Y.-Y-ta we see that u. , - u. = y. - y . . . We also 
3 3 13 13 3 3 

see, however, that u. - u. , = a . -a., and 

u . - u . , = y . - y . . , The reason for this Is that we 

have a completely balanced additive model and u . ^-u^ # 

/ 

is equivalent to u. .-u. . . . 

13 13 


Let us test H : a, = a„ = a_ = a, or 
o 1 2 3 4 


0 1 -1 0 0 0 0 0 


H : 6 3 = 0 1 0 -1 0 0 0 0 6 = 0 

o 

01 0 0-1000 


T 

where 6 = [u a, a. a_ a. y. y, y,] . The appropriate 

X 2. o 4 X 2, o 

test in the "u" model is H : u. . = u. , , . In matrix 

o 1] x'3 

notation, we have: 


U 11 U 12 U 13 U 21 U 22 U 23 U 31 U 32 U 33 U 41 U 42 U 43 
100 -1 00000000 

100000 -1 00000 
100000000 -1 00 
0100 -1 0000000 
0100000 -1 0000U 

0100000000 -1 0 
00100 -1 0.00000 
001 0 0000 -1 000 


00100000000 -1 
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Prom Section 2.3, we see that to test 


H : A u = 0 , we set B = 

o 


e: 


but B is a 15x12 


matrix of rank 9. This can be seen by observing that 
the first row of A T , minus the fourth row of A T , 
is equal to the first row of 0 T , and so on. After 
eliminating the dependent rows of B , we have 

j' 


B = 


e- 


where 


1 

3 


1 1 1 -1 -1 -1 0 0 0 0 0 0 
111 0 0 0 -1 -1 -1 0 0 0 

111 0 0 0 0 o 


o -1 -1 -1 


or in other words we are testing 


H 


u. 


= u. 


= u. 


= u 


Hence if we wanted to use Eq. (3**0 to calculate P , 


we would use H 


A^u - 0 


Numerical examples of 


this procedure are given in Chapter 6, 
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Let us now consider an example where 1 = 1 ,2,3 >4 ; 
j = 1,2,3; and k = . Suppose we observed 

the data in the following manner: 


Vj 

i\. 

1 

7 2 

3 

1 

y lil 
y 112 


y 131 

2 


y 221 

y 221 

3 

y 31.1 

y .3 2 1 


4 

y 4.11 


y 4 3 1 
y 432 


A blank cell means no observations were made at the 


combination of i and J , Again, the "3" model is 

y. = u+ a. + y . + e . . The "u" model is now 

13k 1 3 13k 


y ill 


1 

0 

0 

0 

0 

0 

0 

0 


y il2 


1 

0 

0 

0 

0 

0 

0 

0 


U 11 

y 13 1 


0 

1 

0 

0 

0 

0 

0 

0 


U 13 

y 221 


0 

0 

1 

0 

0 

0 

0 

0 


u 22 

y 2 3 1 


0 

0 

0 

1 

0 

0 

0 

0 


u 23 

y 3 1 1 


0 

0 

0 

0 

1 

0 

0 

0 


U 31 

y 321 


0 

0 

0 

0 

0 

1 

0 

0 


u 3 2 

y 4 1 1 


0 

0 

0 

0 

0 

6 

1 

0 


1— 1 

y 431 


0 

0 

0 

0 

0 

0 

0 

1 


Aj 

l 432 - 


0 

0 

0 

0 

0 

0 

0 

1! 
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subject to 0 u = 0 or subject to 


U 11 - U 13 - U 4 1 + U 43 ' 0 


u. , - u, _ - + u_„ - u,_ + u_„ = 0 

11 13 22 23 31 32 


Thus If the model Is to be additive (that Is, no inter- 
action) then the must satisfy the above con- 

straints. Again, we will postpone the discussion of 
the construction of 0 T until Chapter 5. 


Let us now consider estimating a^-ou,. We can use 

u. ,-u, , . where u. . and u. , . exist; that is we do 
id i'd 1 d i'd 

not say that u. -u.„ is an estimate of a -a. since u,„ 
a 12 42 14 12 

and u 42 do not exist in the model. Rather, we would 
use u 13 “ u 43 ‘ Let us observe the following. First 
\x # -u 2 # is not equal to ot 1 — oi 2 as in the balanced case. 
This can be seen by observing that 


u 


U u + U 13 


2u + 2 0l + Y x + 3-3 


U 2 • = < U 22 + U 23 )/2 = (2U + 2a 2 + Y 2 + Y 3 )/2 


then 


- - r i ~ Y 2 

“l. - U 2- = ( “l - U 2 + - 2 
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Also, suppose we test H q : = a 2 = = a 4 

Again, we test H : u. . = u. . . in the "u” model, 

o id i'd 


U 11 U 13 U 22 U 2 3 U 31 U 32 U 41 U 43 


, T 

Xu = 


1 0 
1 0 
0 0 
0 1 
0 1 


0 

0 

1 

0 

0 


0 

0 

0 

-1 

0 


■1 

0 

0 

0 

0 


0 

0 

-1 

0 

0 


0 0 

■1 0 

0 0 

0 0 

0 -1 


u = 


Now we augment X T to 0 T and obtain B T , that 
Is 


U 11 U 13 U 2 2 U 2 3 U 31 U 32 U 41 U 43 


B 


1 -1 

1 -1 

1 0 

1 0 

0 0 

0 1 

0 1 


0 0 

-1 1 

0 0 

0 0 

1 0 

0 -1 

0 0 


0 0 

-1 1 

-1 0 

0 0 

0 -1 

0 0 

0 0 


-1 1 

0 0 

0 0 

-1 0 

0 • 0 

0 0 

0 -1 


or 


0 
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After eliminating the dependent rows, we obtain 


0 0 


0 -1 


B = 


-i i 


o o 


1 o 


o o 



o o 


o o 


1000 -1 oo o 


0010 o-io o 


oioo o oo-i 


Thus, in order to test H : a. = a„ = a_ = a. , we 

O JL 2. o 4 

T 

test H : X.u =0 or 

o 1 


U 11 - U 31 “ ° 


V \ u 22 - U 32 * 0 


U 13 ' U 43 ’ ° 


Let us restate the procedure for analyzing 

y . .. = u + a. + v . + e ; : 

° 13k 1 3 13k 

( 1 ) Find 6 T 


(2) u = [I - (X T X)" 1 0(0 T (X T X)" 1 e)" 1 0 T ](X T X)" 1 X ,r Y 
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(3) X T u is B.L.U.E. of X T u 

T 

(4) H q : X u = 0 can be tested by using the 

results of Chapter 2 or Theorem 3.0. 


In concluding this section, let us observe the 
following. First, we have given estimates and tests of 
hypothesis for the "classical" estimates and tests of 
hypothesis; however, there is no need to restrict our- 
selves just to these. We would have tested, for 

example, H : 2u. = u„ in either of the above cases 
^ 3 o 1 • 2 • 

T T 

As we know, we can test H : Xu- £ where X is 

3 O 

arbitrary and £ is known. 


Second, if we are in the completely balanced case, 
we know that to test H : a. = a„ * a_ = a. and 

G X 2 . $ 4 

H : y = y = y , we can use 


SS(ct^) 


(y..v 


12 


SS(yJ = • - 


Ey 2 (Y ) 
12 
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to obtain the numerator sum of squares for the test 

^ 2 

statistics and we would obtain a by 

, 2 I>ii - ss(a i> ' ssti-U 

a = 1 

In other words. Theorem 3.0 says that H q : X t u - 0 

T 

is equivalent to H : 6 $ = 0 . Thus if the computa- 

O 

T 

tional form of SS(6 ) is simple, we could use it to 
obtain SS(X T ). If 6 T is not one of the "classical” 
hypotheses, then simple forms probably do not exist for 
SS(S T ) and we would use SS(X T ) directly. 

The generalization to the N-way classification 
follows similar reasoning and will not be given here. 

3.1.3 The Two-Way Classification with Interaction 

The usual two-way classification with interaction 
model is given by y. = u + a. + y. + <5 . . + e... 
i - 1, • • • ,a; j = 1, • • • ,b ; k = 0,1,2, ••• ,n i; . . In this 
case, the rank of X is the number of nonzero n. . , 
which we denote by p . Also, the rank (XP) is p . 
Hence by Theorem 3.0, 

rank(0 T ) = rank(X) - rank(XP) = p - p = 0 
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Hence 0 T = 0 . Therefore, the appropriate "u" model 
is Y = Xu + e . Let us first consider the case 
n ± > = t ; that is, t observations per population. 
Now the test for interaction in the model is 


H o : 6 ij - h'j - hj- - s i-v - 0 all 1, IV, J. 

j ’ . We know, however, that u. . = u + a. + v +6. 

i] i 3 i j 

Hence H : u. . - u. , . - u. . , + u. , = 0 for all i, 

o 13 1 ' j 1 j ' ij 3 

i ' , j j j* is the appropriate test for no interaction 
in the "u" model. 


Now let us consider u, — -u„ . Now 

1 * 2 * 

- u' 2# = bu + ba 1 + Y. + 5 1 . - bu 


or 


= b(a x - a 2 ) + 6 1# 




If we impose the restrictions 6. =0 for all i’s, 

I 

then u^ # “ U 2 » = a i - a 2 • Therefore, in order to 

test H : a. = a. = • • • = a , we test 
o 1 2 a 3 

H : u. . = u_ .= «»■»= u in the "u" model. An anal- 
ogous result is true for the y^s. Again, we need not 
restrict ourselves just to these three tests. Also, 
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for this simple case, we would use the known computa- 
tional equations to obtain the sums of squares for these 
three hypotheses. 


Let us consider an example where k = 0,l,«»»,n i _. 
Let i = 1,2, 3, 4; j - 1,2,3 , and suppose we observe 
the following: 


i x. 

1 

2 

3 

1 

y lli 
y 112 

y i21 

y i31 

2 

y 2 1 1 

y 221 
y 2 2 2 


3 

y 311 

y 3.2.1 

y 331 

4 

y 411 

y 421 

y 4 3 1 
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In matrix notation, we have 


y lll 


1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 



y 112 


1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 


U 11 

y 121 


0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 


U 12 

y i 31 


0 

0 

1 

0 

0 

0/ 

0 

0 

0 

0 

0 


U 13 

y 211 


0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 


u 21 

y 221 


0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 


CM 

CM 

3 

y 2 2 2 


0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 


1 — ! 

00 

2 

y 3 11 


0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 


u 32 

y 321 


0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 


u 33 

y 3 3 1 


0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 


u 41 

y 411 


0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 


u 42 

y 421 


0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 


u 43 

y 431 


0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 



— _ 


— 










— 



T 

In this case., the test for interaction is H : X u = 0 

o 

where 


X 


T 


U 


u 


u 


u 


u 


u 


u. 


u. 


u , 


u 


U 


11 12 13 21 22 31 32 33 41 42 43 


1-1 0-1 1 0 
1 -1 0 0 0 -1 

10-100-1 

■% 

1 -1 0 0 0 0 

1 0 -1 0 0 0 


0 0 0 
10 0 

0 10 

0 0-1 

0 0-1 


0 0 
0 0 
0 0 
1 0 
0 1 


Now let us consider u 1 . _u 2 .* We see that 


Y. 6 rl + 

U = U 4 - 0t_ + —5- 4 - 

i* 13 


12 


+ 6 


13 
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u 2# = u + a 2 + 


Y + Y <5 +6 

T 1 ' 2 , 21 2 2 


Now 


U l* U 2 


“l - a 2 + 3 


Y. (Yj + Y;,) 


(S 21 + « 2 2 > 


Suppose 6 1 . = 0 ; 6 21 + 6 22 = 0 and y. = 0 , 


1*2* “ 1*2* = a l * a 2 + ~ 2 ~ . Hence if we test 
H q : u - u 2# = 0 , we are testing 


Y 

H : a — a + — 0 

o 1 2 d 


in the "3" model. Now let us consider 


U 11 + U 12 U 21 + U 22 


Here we see 




U 21 + U 22 . , h + Y 2 , 5 21 + { 22 

2 - u + a 2 + ^ + 2 


then 


8^4 



Hence 


"ll + U 12 (U 21 + U 22> 


“l " “2 


6 11 + 6 12 " (<S 21 + 6 22 } 


Now suppose we set 6^ + 6 = 0 and 

^2i + ( ^22 = ^ * Then we have that 


U 11 + U 12 U 21 + U 22 


°1 - a 2 


Thus, if we test 


H : 

O 


U 11 + U 12 


U 21 + U 22 


U 31 + U 32 


U 41 + U 42 


and impose the proper nonestimable conditions, we will 
test H: a_=a„=a_=a. . 

O 1 2 . 5 4 

However, what have we Really done? We have ignored 

all y. with j = 3 • In other words we have ’’thrown 
° i jk 

away" data so as to get a complete design and then ana- 
lyzed it as though j = 1,2 . If we do this, the inter- 
pretation of the results should reflect the fact that 
j = 3 was not included in the analysis. We shall 
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postpone the discussion of the merits of such hypoth- 
eses to Chapter 5* The point is that if such a test is 
desired, it can be done. 


Let us summarize the analysis of this model. 

(1) e T = o 

(2) u = u* = (X T X) _1 X T Y 

(3) X T u = X T (X T X) _1 X T Y 

(4) If H q : X T u = % , then 


ss(x T ,?) = [x T u - c] T (x T (x T x)" 1 x) _1 [x T 5 - a 


This concludes the discussion of classification 
models. Extension to higher-order designs are 
straight forward . 


3.2 Design Models 

In this section we will consider some of the more 
common designs found in classical experimental design 
theory. The designs in this section include the Latin 
Square, Balanced Incomplete Block, and the Split Plot 
Designs. Note that the Randomized Complete Block is 
not included since the analysis of it is identical to 
the two-way without interaction. Also, no structure 
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■will be assumed about the "treatments" since it would 
be a simple matter to perform an analysis given a spec- 
ific structure for the treatments. 


3.2.1 Latin Square 

The model for the Latin Square is 


y ijk 


u + a . + Y . + 6 . + e . . . 

i j k i]k 


We can consider this as a three-way classification 
without interaction and with missing cells. 

2 

We know that there are m observations. Now 
X = I and rank(X) = m 2 . Also rank(XP) = 3m - 2 . 
Hence by Theorem 3.0, 

rank(0 T ) = m 2 - 3m + 2 = (m - l)(m - 2) 


Now from Chapter 2, we know that the degrees of freedom 
for error are n-p+r, where n is the number of obser- 
vations, p is the number *of populations, and r is 
the rank (0 T ). Now n=m 2 , p = m 2 , r= (m-l)(m-2) 
hence the degrees of freedom for error are 

n-p+r = (m-l)(m-2) 


which is a well-known result. 
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Let us consider the following 3 X 3 Latin Square 


Column 


l 


Row 2 


3 


in the "u" notation, we have 


12 3 


A 

/ B 

C 

B 

C 

A 

C 

A 

B 


U lll 

U 122 

U 133 

U 2 1 2 

U 223 

U 2 31 

U 313 

U 321 

U 332 


The rank of 0 i= (m - l)(m - 2) =2 • 1=2 . Now 


where 


1-1 0 0 1 - 1-1 0 1 
1 0 - 1-11 0 0-11 


u 


U ^ u ill U 122 U 13 3 u 212 u 223 U 231 U 313 U 321 U 332^ 


The method of constructing 8 T for the Latin Square is 
given in Chapter 5. Now the appropriate "u" model is 

Y = u + e 


subject to 9 T u = 0 . 
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Let us consider u 1>4 -u 2>> , where u 1# . means we 
average over only those terms in the model, that is 



u. 

1 • • 


U lll + 

U 122 + U 133 




3 


2 • • 


U 212 + 

U 223 + U 231 




3 

Now u. = u 

i 3 k 

+ a . 

i 

+ Y 3 

+ • 

Hence 


u., 

l* • 

" *2 

• • 

“l - a 2 

Thus if we test H 

o 

: u 

! . . = u 2 • • = u 3 . . » thls ls 

equivalent to 

H : 
o 

a i 

= a 2 = 

a 3 , Likewise, 

H : u , = u 

o *1* 

•2* 

U * 3* 

is equivalent to 

V n - n ■ 

= Y 3 * 

And H : 
o 

u ..i - u .. 2 ■ u .. 3 ls 

equivalent to 

H : 
o 

L 

H 

o? 

to 

li 

6 3 . Now 


U _ 

• • 1 


u m + 

U 2 31 + U 321 




3 


CM 

l3 : 


U 12f + 

U 212 + U 332 




3 


U • • 3 


U 313 + 

U 223 + U 133 




3 
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Or in matrix notation, the test H q : u . #1 = u .«2 = u **3 

is : 


1 

3 


1 

1 


-1 0-1 0 1 0 
0 -1 0-11 -1 



0 


Now let us suppose that u is missing. In 

this case, rank(0 ) = 1 and 


9 T = [1 _1 _1 0 1 1 -1 0]u 


where 


u 


= Cu 


u 


u 


u 


u. 


u. 


u. 


122 133 212 223 231 313 321 


U 332 ^ 


In this case 


u 


u^„, + u„„, 
231 321 


and 


u 


U 122 + 


U 212 + U 332 


So that we see 


u 


- u 


6 1 ’ 6 2 + 


(a| + a 3 ) 


a. (Y 2 + Y 3 ) Y. 
_ + _ __ 
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Therefore, H q : u .. 1 ~ u .»2 = ® is equivalent to 


H : 
o 


*2 + 


(0t 2 + Ct 3 ) 


a. 

T 


(y 2 + Y 3 ) 


Y. 

3 


= 0 


in the "8" model. Now consider 


H 


U 122 

+ 

U 2 31 

U 223 

" U 133 

= 0 

U 223 

+ 

U 332 

- U 2 3 1 

" U 321 

= 0 


this is equivalent to 


- 28 


H 


( 


* *3 ' M 1 


°\ 

o( 


or 


H : 
o 


S 1 - *2 ' ° 

6 2 - «3 = 0 


Again we see the flexibility we have in choosing X T 

T '• 

in order to test H : X u = £ . If we have the 

o 

standard Latin Square and the standard analysis is 
desired, then we would use the known computational forms 
for finding the appropriate sum of squares. If we have 
missing data, the "u" approach offers a straightforward 
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method for the analysis. Numerical examples will be 
given in Chapter 6 . 


3.2.2 Balanced Incomplete Block (BIB) Design 

./ 

The model for the BIB design is 


u + a . + y, + e . 

i k X]k 


i = l,***,a, 3 = 1 , • • • ,b . Now k = 1 if 1th treat- 
ment is in the j th block and k = 0 otherwise. This 
is merely a two-way without interaction with missing 
cells. It also reflects the condition where the blocks 
and treatments have been arranged so that o^-a,. is 
estimable. In the "u" model, this is equivalent to 
saying that the populations have been sampled so that 
the appropriate test for main effects is 

H : u. . - u. , . = 0 

o id x'u 


The conditions for a BIB design are 

* 

(i) Every block contains k treatments 

(ii) Every treatment occurs in r blocks 

(iii) Every pair of treatments occurs together in 
the same number (denoted by X) of blocks. 
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The number of populations is bk, and there is one obser- 


vation per population. Hence X = I , From Theorem 3.0, 
we know that 


rank( e T ) - rank(X) - rank(XP) 


= rank (I) - rank(P) 


= bk - a - b + 1 


Let us consider the following BIB design. 


Block 

Treatment 

Number 

1 

1 2 

2 

1 3 

3 

2 3 


In the "u" model notation, we have 
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If we had no missing cells, then the constraints on the 

u . . ' s would be u . . - u . . u . . , + u . . . , - 0 , for all 
id id i ' d i D 1 i'd 

i, i ' , j , j ' . Let us list the constraints assuming no 
missing cells. One set of four linearly independent 
constraints is: 


11 

- U 21 

“ U 12 

+ 

c 

to 

to 

It 

0 

(1) 

11 

- 11 

- u, ^ 

+ 

u „ = 

0 

(2) 

21 

13 


23 



- U „ 

- u _ 

+ 

U _ = 

0 

(3) 

11 

31 

12 


32 


11 

" U 31 

- U 13 

+ 

U 33 = 

0 

(4) 


(The procedure for listing this set is given in 

Chapter 5.) Now u 13 > u 22* u 3i are m ^- ss i n S • Thus we 
must eliminate these from the above equations. 

Subtracting Eq. (2) from Eq. (1) we have 



” U 12 + 

u 22 + u 13 

U 23 


0 

and 

subtracting Eq. 

l 

(4) from Eq, (3) 

we 

have 


- U 12 + 

U 32 + U 13 

- U 33 

= 

0 

or 

U 13 

+ U 22 " 

U 23 + ' 

U 12 




U 13 = 

U 33 + 

U 12 

+ u „„ 
32 


94 



or 


U 22 U 23 U 33 + U 32 


Substituting this into Eq. (1) we have 


U 11 U 21 “ U 12 + U 23 " U 33 + U 32 " 0 


Hence 


8 u U 1;L - u 21 - u 12 + u 23 u 33 + u 3 2 0 


And bk -a-b+l=6-3-3+l=l which is the 

T 

rank of 9 , Hence the corresponding "u” model is 

Y = u + e subject to 0 T u = 0 . 

Let us now consider a number of hypotheses 


(i) 

H : 
0 

u. . 
1 .3 

- u. , . 
i'd 

= 0 

(ii) 

H : 
0 

u, 

1 • 

- u. , 

1 ' • 

= 0 

(iii) 

H : 
0 

u. . 
1 3 

- u. . , 
ID 

= 0 

(iv) 

H : 
0 

u . 
* 3 

- U . , 
* D ' 

= 0 


As before, we mean by u. ., u., . and u. only those 

id * 13 id 

4 

u., f s that exist in the model, and u. and u . mean 
13 i* *3 

we sum over those u. . that receive treatment "i" or 

id 

that are in block j . For example, u 1# = u i;L + u 
and u #1 = u 12 + u 32 , and so on. 
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The corresponding hypotheses in the "3” model are: 


(i) Treatment adjusted for blocks 

(ii) Treatments unadjusted 

(iii) Blocks adjusted for treatments 

(iv) Blocks unadjusted 


A numerical example of the BIB design is given in 
Chapter 6. 


3.2.3 Split Plot Design 

The model for the Split Plot Design is 


y ijk 


= u + 


3 . + a. + 
i 3 


5 . . + y. + t .. + e. .. 
13 k jk 1 jk 


where 6 and e„ k are assumed to be independent, 
normally distributed, random variables. If we consider 
that is fixed, this is nothing more than a three- 

way with interaction. This is assuming there is no 
interaction between a. and y, 1 and that there is no 
three-way interaction. In this case, the appropriate 
"u" model is Y = u + e subject to 0 T u = 0 , where 
e T is chosen so as to indicate that the model is 
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partially additive. For example, suppose we had the 
following Split Plot 


y... = u + B . + a . + <5 . . + y. + t + e . 

13 k l 3 13 'k 3 k 13 k 


where i = 1 , 2 , j = 1 , 2 , k = 1,2 and B i = the block 

effect, a. is the main treatment effect, and y, is the 
3 k 

subtreatment effect. 


Block 1 Block 2 

Treat. 1 Treat. 2 Treat. 1 Treat. 2 


Sub 1 

X 

X 

Sub 1 

X 

X 

Sub 2 

X 

X 

Sub 2 

X 

X 


T 

In this case, we have 8 u = 0 where 


111 

U 112 

- U 121 + 

U 122 

“ U 211 

+ 

U 212 

+ U 221 

U 222 

111 

- u !12 

+ U 121 - 

U 122 

" U 211 


U 212 

- U 22 1 

+ U 222 


Again, we refer to Chapter 5 for the construction of 
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The following table gives the appropriate tests 


in the "u" model 


Source of Variation 

H : X T u = 0 

O 

"Blocks" 

u, = u . , 

x * * x ' • • 

"A" 

u . = u . , 

*3* *D * 

Error a 

u.. - u . . . - u . . , + u . . . , =0 

ID * I'D* id * x ' D • 

"B" 

U , - u . , 

"AB" 

U - U •.! “ U + U . , . . = 0 

• D k • d k • d k ' * d k 

Error b 

CD 

£ 

II 

O 


Note that 


error b = min [Y - Xu] a [Y - Xu] 
6 T u=0 


But since X = I , min [Y - Xu] T [Y - Xu] = SS(0 T ) . 
Now suppose u xll is missing. In this case, 

9Tu = U !21 - U 122 - U 221 + U 222 * '° ’ Hel>e 9 * 18 
obtained by eliminating u 111 for the constraint equa- 
tions given above; however, the hypotheses given above 
no longer apply since u 111 I s no longer in the model, 
As in the case of a missing observation in the Latin 
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Square, there are several hypotheses that seem appro- 
priate. Again we defer discussion of these hypotheses 
to Chapter 5 • 

This ends the discussion of relating the classical 
experimental design models to the "u" model. If we are 
given any design model of the form Y = Xu + e , sub- 
ject to u = P3 or Y = W8 + e , we know by 
Theorem 3.0 how to obtain the appropriate "u" model. 

The discussion so far has been limited strictly to 
design models. Through minor modifications, however, 
we can use the "u" model to describe a regression model 
or a covariance model. This will also be covered in 
Chapter 5. 




CHAPTER 4. THE MIXED MODEL 


4.0 Preliminaries 

We now consider the analysis of the "mixed" 
model — a model in which some of the components are 
fixed and some are random. Up to now, we have assumed 
that all the components were fixed, that is we con- 
sidered u. . as a constant. We will still assume 
id 

that we are in a design model, that is X is a matrix 
of zeros and ones. The regression model and covariance 
model will be considered in Chapter 5. 

In order to analyze the mixed model, we will: 

(1) assume that u is a constant; (2) find the sum 

of squares associated with various hypotheses; 

(3) assume the proper structure on the u_ , that is 

we will assume that u . . ~ N(u* . , a . . ) ; .and (4) find 

1 -D ID 1 D 

the expectations of the sum of squares and then test 

hypothesis or estimate the components of variance. 

This is the classical method of analyzing mixed models. 

Here is an example to clarify these concepts. Suppose 

we have y, - u. . + e . . , where 
ijk 13 13k 

u.. = u + a . + y . + 6 , . 

ID 1 D ID 
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and where 


u is a constant 
a. is a constant 



e. ~ N(0,a 2 ) 
13k * 


We would analyze Y = Xu + e assuming u is a con- 
stant, that is we would test H : A T u = v , etc. Let 
us denote the sum of squares associated with 
H : A T u = v by SS(A T ,v) . If we are in a balanced 

case, then the distribution of SS(A T ,v) can be found 

and a test can be performed. If we are in the unbal- 

2 2 

anced case, we would estimate o and a« and find 

Y 0 

variances of these estimates. 

Since the distribution of SS ( A ,v) is known only 
in the balanced case of the mixed model, we shall not 
discuss the test of hypotheses in detail. The more - 

*r 

■i: 

important aspects of the mixed model analysis are the 
problems of estimating variance components and then 
finding variances of these estimates. We shall con- 
sider these problems in depth. 
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In order to do this, we shall develop a procedure 
for finding expectations, variances and covariances for 
sums of squares. This procedure is designed for sum of 
squares obtained by analyzing the "u" model. 

If we test H : A T u = v , then the form of the 
o 

ip 

sum of squares, SS(A ,v), is 

ss(a t ,v) = [a t u - v] t [a t a(x t x)-Va]- 1 

• [A T u - v] (4.1) 

where A, (X^) -1 and u are defined in Chapter 2. 

In order to find a simple expression for Eq. (4.1), 
we must consider the following theorems. 

T 

Theorem 4 . 0 — Suppose we want to test H q : A u = v . 

If T is a nonsingular matrix, then H q : TA t u = Tv is 

equivalent to testing H q : A T u = v , that is 

SS(TA T ,Tv) = SS(A T ,v) . 

Proof — Prom Eq. (4.1), we see that 

SS(TA T ,Tv) = [TA T u - Tv] T [TA T A(X T X)' 1 A' r AT T ]" 1 

• [TA T u - Tv] 

= [A T u - v]V[TA T A(X T X)’ 1 A T AT T ]' 1 T 

rn /n 

• [A t u - v] 
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= [A T u - v] T [A T A(X T X)" X A T A]" 1 [A T u - v] 
= SS(A T ,v) 

T 

Theorem 4,1 — Suppose we wish to test H : A u = v . 
There is a F such that H q : Y u = 6 is equivalent 
to H ; A T u = v , where r T F = I and 
r T A(X T X)“ 1 A T r = D , where D is a diagonal matrix. 

Proof — By Theorem 1.10, there is a nonsingular matrix 
B 1 such that B 1 A T AB^ = I . Set T^ = B A T . Now by 
Theorem 1.11, there is an orthogonal matrix B such 
that 

.B 2 T^A(X T X) -1 A T T 1 B2 = D . 

Let r T = ® 2 T^ = B 2 B 1 A T . Now b 2 ®i Xs nons i n g u l ar 5 

T T 

hence by Theorem 4.0, H q : r u = r v = 6 is equiva- 

T 

lent to H : A u = v where 

o 

r T r = b b a t abV 
2 1 12 

_ 1 

= B IB* 

2 2 

« I 
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and 


r T A(X T X) _ 1 A T A 


B 2 T^A(X T X) 1 A T T 1 B 2 


= D . 


The theorem is proved. 


T 

Prom here on if we test H : A u = v we will 

o ’ 

T T 

assume that A is such that A A = I and 
A T A(X T X) -1 A T A = D . For if A T did not satisfy these 
conditions, we could find an equivalent hypothesis that 
does. Hence Eq. (4.1) can be written as 

SS(A T ,v) = ( A T u - v) T D" 1 (A T u - v) 


(A T u)V l A T u - 2v T D- X A T u 

+ V T D" 1 V 


Let us partition A and v as follows 



(4.2) 
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Hence 


(A T u) T D~ 1 (A T u) 


/\ m _ n m /s 

u AD 1 Au 


u T [X. 


x: 


T 


u 


1 

d. 


1 


V 


0 0 


0 

0 


• • * • 
• • • • 
• • • • 


S-J 




1 . T ^ 

u A . A . u 

a. x x 

i 


■ e t m 

i=l 1 


Also 




■ 

1=1 1 L J 


and 


v t D“ 1 v = 


A v 2 
1=1 1 
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Henc e Eq , (4.2) bee ome s 


SS(A T ,v) 




(4,3) 


Now if we test H : A T u = 

O 

to 


ss(a t ,o) * 


0 , then Eq. (4,3) reduces 



(4.4) 


We shall denote SS(A T ,0) by SS(A T ), Since we normally 

T 

test hypotheses of the form H q : A u - 0 , the 

remainder of this chapter will give results only for 
this case. The other case, while not difficult to 
obtain, involves rather lengthy algebraic expressions. 


Let us now find the expected values, variances and 
covariances of the sum of squares which is given by 
Eq, (4.4). 
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4.1 Expectations, Variances and Covariances of 

Sums of Squares 

The following theorem provides us the moments for 
SS(A T ) . 

Theorem 4.2 — Let 

ss K) 

and 

M ( A S) - E 

j=l 

where u ~ N(u,R) , then 

(1) E fss 



t 5ir E M 
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(ii) v[ss(*j)] = tx v [‘u S ] 


1=1 d li 


+ 2 


E Ejt? 


i < k 


li d lk 


Cov 


/ \ 2 

/ \ 2 

X*.u ,| 

X* u 

\ li / 

l lk / J 


or 


v [“(*!)] ■ E $ 


1-1 Si 


2 ( x ii R Si)' 


+ 4u T X .X, .RX, .X* u 
ll 1.x li lx 


+ 2 


E E 

i < k 


d . d 
li lk 


Cov 


( x Ii 5 ) 2 >( x Ik G ) 


(ill) Cov , Ss(a^)1 


S 1 S 2 

X s V 1. 1 


L, Is 

i=l j=l 


d_. d , 

ll 2 j 


Cov 


( A li G ) 2 ’( X 2j G ) 2 ] 


(iv) Cov 


( X L G ) • 


^ / m \ ^ 

X 2j u 


= 2 


(*U Ri 2J )‘ 


+ 4u T X .\l .RX. .xj.u 

2j 2 j li li 
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Proof — The proof of this theorem relies on repeated 
application of Theorem 1.8. Now we have that 


E 


SS | A * ) 


■ £ 4 4:. ; ] 

i=l 11 L J 


From part (i ) of Theorem 1.8, we see that 
12 


T ^ 

; a. .u 

ii 


= E 


Trace RA..A:f. + u T A ..a'J’.u 

I li lij li li 

Trace aIF.RA... + u T A..A^.u 

I li lij li li 


m rp m 

A, , RA. . + u A. . A: .u 

li li li li 


Hence part (i) is established. 


Now 


7 [ ss (;•!)] - L^" V [ X U“] 


+ G 


i=l li 


where 


C - 2 




i < k 
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Again by Theorem 1.8, part (ii), we have that 

vfx^.uV = 2 Trace [rA . A^" . 1 + 4u T A. . A* .RA. . X T . .u 

\ lx / [_ li lxl lx lx lx lx 

= 2 Trace RA. . A^ .RA. . A^ . 

| lx lx l.i IxJ 

+ 4u T A .A^.RA. .A^.u 
lx ll ll lx 

= 2 Trace A^RA, . A^.RA. . 

I lx lx lx lxl 

+ 4u T A .A^.RA A^.u 
lx lx lx lx 


= 2 (a^ . RA . ) + 4u T A .a'F.RA A?\u 

\ lx hi lx lx lx lx 


The expression for C will be given later. Hence 
part (ii) is established. Now part (iii) is established 
by the linear property of covariance. By Theorem 1.8, 
part (iii), we see that 


Cov 


.( X Ii“f-( X 2 j ;: n = 2 TraCe [ RX li X Ii RX 2 j X 2 j] 


+ 4u t A .a'J’.RA. .A^.u 

lx lx 2j 2 3 


= 2 Trace A^ .RA . A^ . RA . 

2j lx 23 IxJ 


+ 4u T A . A^RA. .A* .u 

lx lx 2j 2j 
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cov [(^)\(^) 2 ] 2(x* 3 HX_ l± )‘ 


+ *; jU 


Hence part (iv) is established. Now C in part (ii) 
is seen to be 


* 2 S 2d li 1 d k [ 2 ( X lk RX ii) + 4u \i X Ii RX ik X ik u ] 


i < k 


Hence the theorem is proved. 


Let us discuss some of the computation aspects of 
Theorem 4.2, Let b^ = RX 1;L , b 2 .. * RX 2 _., f li = uTx ii 
and f = u T X . . These quantities are needed for 


SS ( A i) and E SS (a*) , 


that is 


B [ SS ( A I)] - £ 377 [ x xi b i± + f ii]‘ 


But now we see that these quantities are needed for 


r j^SS (‘I)]- that is 

v H a i)] ■ £ 


L-4 d 2 

i=l a li 




.b ) + 4f* X*.b. . 

i 11/ li li li 




.b.J + 4f . 

i lk 


i < k 
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and 


Cov 


-(*:)•“(*;) • E E 


S 1 S 2 


1=1 j=l 


d li d 2j L 




+ 4f 


li f 2 j X li b 2 j] 


Once the expectations are found, the effort to find the 
variances and covariances is minimal. 


Before returning to the analysis of the mixed 
model, let us note that in the completely fixed model. 


there is no need to find E 
Theorem 4.2 because SS 


SS (a* j and ¥ Ss(aJ) 


by 


( A i)/ S i 1 


is a noncentral chi- 


square and the moments can be expressed in terms of the 
noncentrality parameter. 


From Eq. (2.9), we see that 

u = u + A(X T X) - 1 X T e 


Now we will assume a structure on the u. . 

S 1 3 

T 

we will write Y = Xu + e subject to 0 u 


that is 
0 where 


u 



i=l 



V.b . 
3 3 


(4.5) 
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and where 


U. is a pxs. matrix associated with a. 

X ^ X \ x 


a. is an s.xl constant vector 
x x 


V. is a pxs. matrix associated with b. 
3 3 3 


b. is an s.xi random vector such that 
3 3 

b . ~ N 

3 




e ~ N ( 0 j cr I) 


e, bj and b.., for j ^ j ' are mutually 
independent . 


Now we see that 


k. 


A 

u 


y U.a. + y V.b j + A(X T X)-V- 

3=1 ^ ^ 


i=l 


Hence 


E[u] = U i a i 


i=l 


V[u] = 


3=1 


V.vTtf? + A(X T X)" 1 A T a 2 
3 3 b 
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Let us now show that SS(A T ) and a 2 are inde- 
pendent where 

a 2 = (Y - Xu) T (Y - Xu) 
n - p + r 

e T We 

n - p + r 

where W - I - XA (X T X) " 1 X T , Now we know that 
u - u - A(X T X) -1 X T e , Let z - We . Hence u-u and z 
are independent since W T A(X T X) _1 X T = 0 . Thus 
Cov [u - u, z] = 0 . But 

Cov [u - u, z] = Cov [u,z] - Cov [u,z] 

But Cov [u,z] = Cov [u,We] = Cov [u,e]W = 0 since we 
are assuming by the structure of u that u and e 
are independent. Hence Cov [u s z] = 0 . But u and 
z are both normal, hence u and z are independent. 
Hence SS j and a 2 are independent. This fact will 
be useful when we are finding variances of the estimates 
of components of variance. This results says that the 
sum of squares associated with H q : A t u - 0 is inde- 

a. 2 

pendent of a no matter if the model is fixed or 
random, or if we have a balanced or unbalanced design. 
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At this point, there are sufficient tools to find 
estimates of components of variance and the variance of 
these estimates. This can be done by using Theorem 4.2 
and assuming the proper structure on u , that is u 
has a structure as given by Eq. (4.5). 

In order to provide a better understanding of 
these concepts, we shall consider a number of specific 
examples . 

4,2 Mixed Model Examples 
4.2.1 The One-Way Classification 
For this model, we assume 
y . , - u . + e . . i = 1, • • • ,a,a + 1 : 

i] i 1 3 

J “ 

where - y + b.^ and b i ~ N ^0 ,a^ j . In terms of 

Eq. (4.5), we have that u = yl + V b . Now suppose 

2 ** 

we wish to estimate . We begin by assuming a com- 
pletely fixed model. Let us test H : u. = u. . . 

Then SS(A T ) = £ • But now from Eq. (2.9), 

we know that 

u - yl + V^b + A(X T X) - 1 X T e 
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Now for the one-way classification, we know that 


V = I and A - I , hence 


u = yl + b + (X T X) _1 X T e 


(4.6) 


Thus E[u] - yl , and 


V[u] = I » e 2 + cr 2 (x' r X )“ 1 

b 


Therefore, we see by Theorem 4.2 that 

e[ss(a t )] = ^ ^ ( x i RX i + y 2 (i t x.) 2 ) 

i=l 1 


But IX. = 0 since we are testing 
x 


H : U- = 

O X 2 


... = u = u 

a a+1 


Also we see that 


A T RA. = oVa. + aT(X T X)~ 1 A , a 2 
xx b x x x x 


a 2 + d . a 2 
b t x 


since A T (X T X) _1 A = D . Therefore 


a 


E[SS(A T )] = acr 2 + i 


i=l 
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Now let 


k 


o 


£ 

•? 


1 

d. 


^ 2 

Let us use o from Theorem 2.2 as an estimate of 

2 2 
a . Hence an unbiased estimate of a, is 

D 


-2 _ SS ( A 1 ) - aa" 1 

b ' k o 


(4.7) 


Now 

v|a 2 ] = [VCSS(A T )] + a 2 V(cx 2 ) 

- 2a Cov [SS(A T ) ,a 2 ]]^k 2 


and 


V[SS(A T ) ] 



* * Z ‘ 2 (**,)' 

i < J 1 

where 

xTRX. = a 2 + d.a 2 
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and 


A TRA . = 0 

i 3 


V[SS(A )] = 


Z + d ^ 2 

1=1 i 


1=1 Q i 1=1 1 


+ aa 


2 k .a 4 + 2k a 2 a 2 + aa 4 
lb Ob 


hence 


1 Ls d 2 
1=1 a i 


Z^ 


- (a * 1)>q ' ~ x 2 (n - (a + 1)) 

o 


no 4 ) = 
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Since SS(A T ) and a 2 are independent, we know that 
Cov [SS(A T ),a 2 ] * 0 . Thus 



Let us now consider the special case where we have 
an equal number of observations from each population, 
that is t observations per cell. In this case, 

X T X - tl and D = (X T X) _1 *= — ■ I . Hence we see that 
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:vf 4<A> 2 

— + — £— + 
a at 


2 ( n - 1) a 4 


(n - (a + 1) )u a 


which is the classical result for the variance of o' 


in the balanced case. 


Now it is easy to see that Eq.' (4.7) represents an 

2 T 

unbiased estimate for o^ for any A such that 

ip 

A 1 = 0 . Thus it is possible to generate an entire 
class of unbiased estimates of c b . And Eq. (4.8) is 
the general expression for the variance of such an esti- 
mate. Now if A 1 ^ 0 , then it would be necessary to 
obtain an estimate of y since E[SS (A )] would contain 
a y . This does not pose any problem since Theorem 4.2 
is completely general. 

4.2.2 The Two-Way Classification Model Without 
Interaction (Random) 

For the moment, we will restrict ourselves to 
finding estimates of the components of variance. We 
will assume the model is Y = Xu + e subject to 
0 u - 0 , where 0 u = 0 implies an additive model. 

We will also assume that 

u — yl + V^a + V 2 b 
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Suppose we wish to test (1) H : u = v. - u 

O X j 

and (2) H q : u = u >2 . Let A 1 denote the appro- 
priate hypothesis for (1), and A 2 denote the appro- 
priate hypothesis for (2). 
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Now 


A(X T X) - 1 A T = 


.3174 

.1826 

. 0506 

-.0843 

.1124 

-.0225 

,1826 

• 3174 

-.0506 

.0843 

-.1124 

.0225 

,0506 

-. 0506 , 

.1629 

.0618 

.0843 

-.0169 

-.0843 

,0843 

. 06 X 8 

.2303 

-.1404 

.0281 

.1124 

-.1124 

.0843 

-.1404 

.3539 

.1292 

-,0225 

,0225 

-. 0 X 69 

.0281 

,1292 

.1742 


and 



.2074 

.2074 

-.5703 -.5703 

• 3629 

.3629 


.5388 

.5388 

-,0898 -.0898 

-.4490 

-.4490 

1 

4083C1 -1 1 

-1 1 -1] 




and 

a^a(x t x) - 1 a t a 1 = 
a’^A(X t X)~ 1 A T % 2 - .4045 

Thus 

U = yl + V a + V 2 b + A(X T X)“ 1 X T e 


.333 0 

0 .449 
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and 


E[u] = yX 

V[u] * V_vV + VV^cr 2 + A(X T X) _ 1 A T a 2 
1 1 a 2 2 b 


and 


X^tVCullA^ 

X^ 2 [V[u]]X 12 

xL[V[u]]A nl 


x L Y i Y l\A 


^2 V l V I X 12°a 


,T Tr Tr T, _2 
X 21 V 2 V 2 X 21 0 b 


+ d o 


+ d 12° 


+ d 2i a 


2 


2 


2 


Let 

b li = X ii V i = 2 t- 2 0T4 -.5703 .3629] 

b l2 = X i2 V i = 2C.5388 -.0989 -.4490] 

b 2l = X 21 V 2 = 3C . 4083 -.4083] 

and 

= [.4046 -.4046 .3035 

-.1349] 


f 2i = X^A(X t X)^A t 


-.5056 .6742 


124 



Thus from Theorem 4.2, we see that 


E 



and 


E 



and 



d 


11 


b^ b o 2 + d a 2 
11 11 a 11 


12 


bT 12 b 12 ( ’a + d 12 b2 


b ll b ll b 12 b 12 

— — ’T -X 


11 


12 


+ 2o‘ 


2 a 2 + 


(l73 + Tw) a i 


2a + (6 + 4 . 


2a 2 + 10.45a 2 

a 


h T 

2121 „2 , „.2 

— a + a 

d 21 b 


3 „2 „ 2 

.4045 b + a 


a 2 + 7.417a 


b 


11 


K 

. V 


2 2\ ^ 
. b a ^ + d a 
ll 11 a 11 


d 


( b L b 12 ff a + d 12 (?2 


12 
1 1 


)1 


d ll d 12 


_ 2 ( b Il b 12°a) 2 ] 
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= 4a 4 + 2^6 2 + ( 4 . 45 ) 2 + )g^ 

+ 4(10. 45)a 2 a 2 

cL 


= 4a 4 + 111.6a 2 + 4l.80a 2 a 2 

a a 


V[SSU 2 )] = 2c 4 + a 


21 


21 


2a 4 + 4(7.4l7)a 2 a 2 + 2(7.4l7) 2 a 4 

D 


2a 4 + 29.66 a 2 a 2 + 110.02a 4 

b b 
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cr ^ 



and 



= 2a 4 ( . 9938 ) = 1.988a 4 


Now 



ao 4 / /v2 

and since V\a ) = a /7 j the variances of a and 

cl 

0 * can be easily found. Let us now outline the pro- 

2 2 

cedure for finding estimates of a and a, for 

Cl C 

I T T 

arbitrary hypothesis. Suppose we have A^. and A 2 . 
Then 

E [ SS ( A l)] = k l° 2 + k 2 '« + k 3°b 

-[-(‘I)] - V 2 + Va + Vb 
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Now if we set 


k l 

k 2 

k 3 _ 

a 2 ’ 


ss(a^) 

k 4 

k 5 

k 6 

a 2 

a 

= 

ss(a^) 

1 

0 

0 



Residual 


Then 


c 2 


k l k 2 k 3 

-1 

SS (a* ) 

o 2 

a 

= 

k 4 k 5 k 6 


ss(a^) 

ii 


_1 0 0 _ 


Residual 


and the covariance matrix of the estimates can be 
found from the covariance matrix of SS j , SS^A^ j and 
residual. The values of k i and the covariance matrix 
of the sums of squares can be obtained by Theorem 4.2. 


2 

Let us observe that a does not have to be estimated 
by residual. Such an estimate, however, has many 
desirable properties , i . e . , central chi-square ,' unbiased 
and so on. 

I, 


4.2.3 The Two-Way Classification Without Interaction 
(Mixed) 

Let us now assume that 

u = y 1 + U ct + V b (4.9) 
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where y, a, , V - 9 b are as defined above . Before 
we analyze this model, let us consider the Idea of com- 
bining unbiased estimates. Suppose t^ • • • ,t n are 
unbiased estimates of 0 and that V is the covari- 
ance matrix of the t^s. We would like to find a set 
of a. 's such that T'.a.t, is an unbiased estimate of 0 

X ' X X 

and v|^a i t i j is a minimum. Let t T - (t x , • • • , t n ) 
and a = (a^, • • • , a ) , then the problem is to mini- 

T T 

mize a Va subject to E[a t] = 0 or subject to 

T 

a H ~ 1 where £ is a vector of all ones. The choice 
of a , as given by Theorem 1.13, is 

„ _ V _1 £ 

^ T - 1 

SL l V l 


Now suppose V is unknown. If we can estimate V , 

A /X 

say by V , so that V and t are uncorrelated, then 
t i T V - ^t 

a t = -—rr - — is an unbiased estimate of 0 since 

£ T v _1 £ 


E[a T t ] 


= E 


T - 1 

& r v 1 

T A - 1 

ii i 


E[t] = E 


T -1 

j/V _ 

ip A -i 

a v a 


£ 0 


* E 


£ T V~ 1 £ 
£ T T -1 £ 


0 = 0 


The merits of such an estimate will not be discussed 
here. 
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Let us now analyze the u structure as given by 

Eq. (2.9). In order to obtain estimates of a, and 

b 

2 

a , we can use the procedure outlined above. The 
interest now is in estimates of the constants. For 
example, suppose we wanted to find an unbiased estimate 

of in the mode1 ' There are two methods by 

which we can do this. First, find so that 

X^u = yX^l + X^I^a + X^V^b “ a i ~ a 2 5 and then use 

X^u as an estimate. Second, find a X 2 so that 

r T A i t a 

E X u = a 1 - « 2 and use X 2 u as an estimate. The 
difference between the two methods is the following; 

In the first case, X^u = a 1 - a 2 + ^A(X T X) -1 X T e and 
E^X^uJ = a ! “ a 2 i while in the second case 

X 2 u = a 1 - a 2 + X^b + X 2 A(X T X)" 1 X T e , and 

E t X 2 a ] = “l - “2 + »^<E(b)) = 

since E(b) = 0 . Thus in the second case, 

X 2 U ^ a 1 ” a 2 = a i ~ a 2 * Hence A^U 

and X 2 u are two estimates of a 1 ~« 2 . Let us consider 

the following set. * 

Let S = {X/X T U 1 a = a 1 “ a 2 and YA T 1 = 0} then if 

X.eS, 

E jx^fuj = E^(a 1 - a 2 ) + X^Vjb + X^A(X T X) _1 X T e] 
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E X . u = a, 

i 1 


V X ju 


X T V V.X.) o 2 + d.a 2 

x 1 1 1/ b X 


2 2 T ^ 

If we can estimate a, and a so that A.u is uncor- 

b l 

^ 2 ^2 

related with a, or a , then we can obtain an esti- 
b 3 

mate as described above. For illustration let us 

consider the same example as described in Section 4.2.2. 

T 

Now one choice for X^^ would be 


X^ = [1 0 -1 0 0 0] 


and X ^ could be 


X * = [1 0 0 -1 0 0 ] 


a l " a 2 + X:[A(X T xr :L X T e 


X^u = a x - a 2 + b x - b 2 + X 2 A(X T X) -1 X T e 


and we see 


that E^uJ = e [ X 2^] = a i “ 

v [ x i“] ■ v 2 


a 2 . And 


2 2 

2 °b + d 2° 
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and 


Cov 


/, Ta , t~ 
[ X 1 U > X 2 U 


= A^A(X t X)' 1 A t X 2 0 2 - ca 2 


Let 


a 


(Y - Xu) (Y - Xu) 
n - p + r 


It has been shown that u and hence X u is independent 

^ 2 T 

of a * Now let A be any hypothesis such that 
A t 1 = 0 and A T U 1 = 0 , then 


Now 


E[SS(A T )] 


S 


£ 


s!v.v!s . 

J- A - L j. 

d. 



sa 


2 


-2 _ SS(A t ) - sa 2 

°b " k 


where 



. J _ A k-2 
d. 

3 


T T ^ ^ 2 

Now if X.eS, then X.u and a, are uncorrelated. This 

i i b 

can be seen from the following: 
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Cov 


[«] - £ 


Cov 


i*l 


rn /\ 

xiu, 

x 


/MTI m A 

u T 6 , 6 T u 

1 2 

d.k 

3 


- I Cov 


But Cov 


^ 2 1 
‘i U ’ ° J 

[x*U, ct 2 J = 0 and by Theorem 1.12 

r WA A>r rr 1 ^ rn m 

Cov X;u, u6 . <5 ;u = 2X.V6.<S a u 

LX x xj XXX 

where u = E[u] = yl + U i a . But 6^u = 0 . Hence 

Cov X^u, u i 6 i 6 tu = 0 . Hence Cov f X 7u, a^j = 0 . 


Now set 


V = 


d o ‘ 
l 

c a 


oo 

*2 *2 

2 < + v 


and 


a = 


v- 1 * 

S, T V _1 & 


then 


a 


rp / 

X^u ] 


X^ u 


is an unbiased estimate of o^-o^. The choice of A 

T T 

and X^, X 2 and so on will not be discussed here, 
us note that this technique is used by many authors 
when analyzing the Balanced Incomplete Block Design 


Let 
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when treatments are fixed and blocks are considered 

T 

random. For this design, they usually choose A and 
T 

X 2 to represent the so-called interblock and intrablock 

T T 

estimates. Also they choose A so that A repre- 
sents the hypothesis of treatments adjusted for blocks. 


In other words, they use t, « u. . - u. . . as one 

1 i 3 

estimate of o^-o^ , . The other estimate they use is 
t = u' - u !,. where u! . = £n .y /(r - A) . Now 

Z X X X . -L J J 


due to the construction of the BIB design, u].— is 

independent of u^-tL.^. The estimate of a £ is 

obtained by testing H q : u^ = u i j • » and calculating 

2 

the estimate of a, in the usual manner. Their esti- 


mate of cu-ou, is 


where 


t = 


Vl + *2*2 


V 


1 

V 

a 2 


Z V 





and where V is a diagonal matrix with elements that 

~ 2 *2 A 

are functions of o and . Note that this pro- 

cedure is merely a special case of the procedure 

T T 

described above, that is the hypotheses and X 2 

associated with u. .-u. , . and u! -u! . are elements of 
S and A t 1 = 0 and A T U 1 a = 0 . Since we are not 
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restricting the estimates of a^-oL, to be independent, 
it is possible to obtain other unbiased estimates than 
those usually suggested by most authors. A numerical 
example will be given in Chapter 6. 

4.2.4 Point Estimation in the General Mixed Model 

In the previous section, we restricted our discus- 
sion to the two-way classification without interaction. 
The methods described there are completely general and 
can be extended to other models. In this section, we 
shall consider a general mixed model and discuss 

T 

another procedure for estimating X u. let Y = Xu + e 
subject to 0 T u = 0 , Now suppose e ~ N(0,V) . If 
V is known, then the B.L.U.E. of u is given by 

u = [i - (x T v~ 1 x) - 1 e(0 T (x T v" 1 x)“ 1 0)" 1 e T ][x' r v" 1 x ]~ 1 
x t v -1 y 

Hence the B.L.U.E. of X T u is given by X T u. In most 
cases V is not known. When this happens it is possible 

A ; t 

to replace V by V , an estimate of V . If we want 

A /S 

u to remain unbiased, we can choose V so that V 
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and e are uncorrelated, that is suppose we replace V 

A 

by V , then 

U* = [i - (x T v~ 1 x)' 1 e(e T (x T Y _1 x) _1 e)" 1 0 T ] 
[X T V -1 X] -1 X T V" 1 [Xu + e] 


or u* = u + A*e where 


A* 


[i - (x T v~ 1 x)“ 1 e(e T (x T v -1 x)" 1 e)‘ 1 e T ] 


[x t v" 1 x]- 1 x t y _1 


Now if V and e are unc or related, then 

E[A*e] - E[A*]E[e] = E[A«] *0=0 

Thus E[u*] = u , An example of this approach is 

given in Chapter 6. 



CHAPTER 5. OTHER TOPICS 


5 . 0 Pr el iminar ies 

In this chapter, we shall consider other topics 
related to the analysis of linear models. These topics 
include regression, covariance, and a discussion of 
tests for main effects and interactions. A section on 

T 

the construction of 0 also is included. 

5.1 Regression 

The "u" model as defined in Chapter 2 is the clas- 
sical design model. We could have started with the 
general regression model and then observed that the 
"u" model is a special case; however, since we wanted 
to emphasize the use of the "u" model in experimental 
design, we chose to start with the "u" model as 
defined in Chapter 2 . We shall now define the general 
regression model. 

Let e ~ N( 0 ,o 2 I ) and let Y = Xu + e subject 

T 

to 0 u = 0 . We no longer consider u . . as the 

id 

T 

mean of the (ij ) th population and X X is not generally 
diagonal. Likewise, u* = (X T X) _1 X T Y is no longer 
the "cell" means. The theory developed in Chapter 2 


137 



is completely general, and by changing the inter- 
pretation of u, u, u* , and X , we can use the 
results developed there. The classical regression 
equation is 

Y * Xu + e 

Usually, there are no restrictions placed on the 

parameters. Hence we see that u = (X T X) -1 X T Y . If 

H : A T u = £ then 

o 

p _ [A T u - 0 T (A T (X T X) -1 A)" 1 [A T u - a 

P - XI 

sa 

where 

p _ y t [i - x(x t x)~ 1 x t ] y 

a " n - p + r 

These are all well-known results. 

If there are restrictions placed on the param- 

T A ^ 2 

eters, that is 6 ^ 0 , then u, a and P can be 

found by the techniques described in Chapter 2. Since 
the analysis of these models is well known, they will 
not be discussed further. 
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5.2 Covariance Model 


In this section, we will consider a special ease 
of the regression model, that is 

m 

y . ... = u ... . + 3 z . . + e . .. 

" 13 k X 3 / p x' 3 .kp xjk 

P=1 

T 

subject to 0 u = 0 . Or in matrix notation, we have 

Y » Xu + Ze + e (5.1) 

subject to 0 T u » 0 where 

X is the nxp design matrix of rank p 
u is the vector of u . . 

Z is the nxm matrix associated with 3 
3 is the vector of covariates 
e ~ N ( 0 , a 2 1 ) . 


Let us consider the density of Y 

_rj 

f (y;u,3,a 2 ) = (2ira 2 ) 2 

I 

. EXP }- il ~ Xu ~ Z3) T (Y - Xu - Z 3) 

( 2a 2 

T 

subject to 0 u = 0 . We will find maximum likelihood 

2 

estimates of u, 3, cr As is customary, we maximize 
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In f (y;u, 3,0 ) - 2 ln ( 2 Tra 2 ) 

(Y - Xu - Z3) T (Y - Xu - Zg) 
20 2 
T 

subject to 0 u = 0 . The Lagrangian function is 

L - -|ln (2tt0 2 ) - ( - Y - Xu - 

d 20 2 

, 26 T 0 T u 


3L 

30 ; 


n , (Y - Xu - ZB) (Y - Xu - ZB) 

2 + 4 

20 ^ 20 4 


26 T 6 T u 


= 0 


( 1 ) 


3L 

3u 


2(-X T Y + (X T X)u + X T Z3 + es) 
20 2 


= 0 


( 2 ) 


3L 

33 


2(-Z T Y + (Z T Z)3 + Z T Xu ) 


= 0 


20' 


(3) 


3L 

36' 


26 T u 


= 0 


(4) 


Now from (1) and (4) we get 


~2 _ [Y - Xu - ZB] [Y - Xu - ZB] 

n 


Prom (2), we see that 

u = (X T X)"- l X 1 Y - (X i X)“ x [X x Z3 + 86] 


140 



But from (4 ) we see that 0 T u = 0 hence 


0 


e T u = e T (x T x) 1 x t y - e T (x T x) 1 x T z§ 
- (0 T (x T x)~ 1 e)6 


or 


6 = (0 T (x T x)" 1 e) _ 1 0 T [ (x t x) - 1 x t y - (x T x)" 1 x T ze] 

Hence 

u = (X T X)“ 1 X T Y - (X T X ) ~ 1 [X T Z0] 

- (X T X)~ 1 0[0 T (X T X)“ 1 0]~ 1 0 T (X T X)" 1 X T Y 

- (X T X) -1 e[0 T (X ,r X) _1 0] -1 0 T (X T X) _1 X T Z3 

* [I - (X T X) _1 0(0 T (X T X)“ 1 0) _1 0 T ]u* 

- [ I - (X T X)" 1 0(e T (X T X)" 1 0)“ 1 0 T ]Y* 

where 

u* = (X T X) _1 X T Y 

y* = (x T x) _ 1 x T ze 


Let | 

A = [I - (X T X ) -1 0 ( 0 T (X T X ) -1 0 ) -1 0 T ] 

then 


u 


A[u* - Y *] 


(5.2) 



Now from (3) , we see that 


(z T z)g = z t y - z T xu 

= Z T Y - Z T XA[u* - Y*] 

= Z T Y - Z T XAu* + Z T XA(X T X)" 1 X T ZB 

[Z T Z - Z T XA(x' r X) -1 X T Z]3 = z t y - Z T XA(X T X) _1 X T Y 

= Z T [I - xa(x t x) -1 x t ]y 

z t [i - XA(X T X) _1 X T ]Z§ = Z T [I - XA(X T X) _1 X T ]Y 


or 

3 W [ Z T ( I - XA(X T X)" 1 X T )Z]" 1 Z T [I - XA(X T X)“ 1 X T ]Y 

(5.3) 


Now let 


£2 _ [Y - Xu - Z3] T [Y -s,Xu - Z3] 

n - p + r 


(5.4) 


By following similar arguments as .in Chapter 2, 
we can show that u, 3, o 2 are minimum variance 
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unbiased estimates. Now suppose we wish to test 
H : A T g = 0 . Then 


SS(A T ) 


(A T g) T 


~ V[A T §]' 

2 

L a 


-1 


(A T 0) 


( 5 . 5 ) 


Now 

V(A p) = A i V(g)A 

and 

V(I) = [Z T (I - XA (X T X) -1 X T )Z] _1 a 2 
Now let us consider V(u). 

V(u) = A[V(u* ) + V(y«) - 2 Cov (u*,y*)]A T 

Now 


V (u* ) 

= a 2 (X T X) _1 

V(y*) 

= (X T X)" 1 X T ZV(3)Z T X(X T X) _1 

Cov (u*,y*) 

= Cov [(x t x)“ 1 x t y, (X T X) -1 X T Zg)3 


= (X T X) _1 X T [Cov (Y,g)]Z T X(X T X)~ 1 


= (X T X) _1 X T [I - x(x t x)“ 1 a t x t ] . 


• ZV(g)Z T (X T X) _1 


= [(X T X) _1 X T - (X T X) _1 A T X T ]ZV(g) 
• z T (x T x) _1 
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But 

ACCov (u*,y*)]A T = - AitfxVhVi 

• zv(B)z t (x t x)" 1 a t 

■« [a(x t x) _ 1 x t - A(x T x)' 1 x T ]zv(e) 

• z’ e (x t x)" :l a t 

» 0 

Thus 

V(u) = A (X T X)'V + A(X T X)" 1 X T ZV(e)Z T X(X T X)“ 1 A T 0 2 

0 , then 

(5.6) 

Therefore Eq’s. (5.5) and (5.6) provide us with direct 
tests of hypothesis for testing H : A T 3 - 0 and 

H : A t u = 0 respectively. 


Now suppose we wish to test H q : A u = 


A t (V(u))" 1 A 


(A t u) 
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Let us consider the following example taken from 


Os tie [24] . 


Gains in Weight (Y) and Initial Weights (X) of 
Pigs in a Feeding Trial 

Treatment 


Total 


1 2 3 4 


x ! Y 

I ! 

X 

Y 

X 

Y 

X 1 Y 

' 

30 j 165 

24 

180 

34 

156 

4l j 201 

27 j 170 

31 

169 

32 

I 89 

32 1 173 

20 | 130 

20 

171 

35 

138 

30 j 200 

21 ■ 156 

26 

161 

35 

190 

35 : 193 

33 j 167 

20 

180 

30 

160 

28 j 142 

29 j 151 

25 

170 

29 

172 

36 1 189 

'* 

160 i 939 

146 i 

1031 

W5 

1005 

202 | 1098 


The model is 



u , + . + e , . 

i 13 id 


Now A = I , (X T X) = 61 3 Z T X = [160 146 195 202] 

Hence 

Z T [I - xa(x t x)~ 1 x t ]z = Z T Z - (Z X)U T Z) 

= 21319 - 20957.5 = 361-5 
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and 


Z T [I - XA(X T X)~VlY 


Z t Y - ( zT x)(x t y) 


120253 - 119756.17 


= 496.83 


Thus , 3 P 1.374 and V(3) « a 2 (36l.5)~ 1 . 


Therefore, If we test H q : 3*0, then 

SS( 3 ) = 361.5(1. 374 ) 2 = 682.5. 


A (u* - y* ) 


* y - \ x T zi 


156.50 
171.83 

167.50 
183.00 


160 

r- 146 (1.374) 
195 


u = 


119.85 

138.39 

122.83 

136.73 



Suppose we wished to test H : u, * = u = u or 

0 12 3 4 

T 

H : A u = 0 where 

o 


Now 


V(u) = 


‘1 -1 0 O' 

10-10 


Li 

0 

0 -1. 

J 


" 2.13 

1.79 

2.40 

2.48 

__ 

1.79 

1.81 

2.19 

2.27 


2.40 

2.19 

3.09 

3.03 


2.48 

2.27 

3.03 

3.30 


Hence 


SS(A t ) 


= ( A ^) ^A T y(u)Al 


-1 


[A T G] 


1609.6 


This concludes the discussion of covariance. 
Eq’s. (5.2), (5.3), (5.4), (5.5), (5.6) provide the 
information necessary to analyze a general covariance’ 
model as described by Eq. (5.1). 

5.3 The Construction of 0 T 

In this section, we consider two methods of 
constructing 0 T . The first uses Theorem 3.0, that 
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5 


then 


is, if we are given the " g" model Y = XP3 + 
we choose 0 so that 

(1) 0 T P = 0 

(2) rank(0 T ) = rank(X) - rank(XP) , 


This can be done by finding the linearly independent 
rows of I-PP + . However, this requires computing P + , 
which can be time consuming if P is a "large" matrix. 

The second approach can be used for most of the 
classical experimental design models. Here we must 

T 

know the functional form of 0 u. For example, in the 
two-way without interaction we know that the u. . 
must satisfy 


u . . t- u , , . 

ID i D 


U . . , + U . , . , 

i j l] 


0 


for all i, i 1 , j, J 1 . 

T 

We will now give rules for finding 0 for the 
two-way classification without interaction, and the 
three-way classification with interaction. First, we 
will assume no missing cells and then consider the case 
when there are missing cells. 
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5.3*1 The Two-Way Classification Without Interaction 

Let us consider the following model. 

y. = u . . + e . . , 
xjk 13 ijk 

i = 1 , 2 , 3, 4, 5; j « 1, 2 , 3 , 4; k * !,'••, n sub- 

j ect to u.. - u . , . - u . . . + u . , , , = 0 for all 
i » 1 ' , j , j 1 • 

T 

We can generate 0 in the following manner. 

(1) Write u lx - Ujl - u XJ + Ujj 

(2) Let J vary from 2 to 4 with I * 2 4 

(3) Let J vary from 2 to 4 with 1=3. 

(4) Let J vary from 2 to 4 with 1=4. 

(5) Let J vary from 2 to 4 with 1=5. 

In computer notation, we would have 

LI = 1 

L2 = 1 

t. 

DO 1 I = 2,5 

DO 1 , J = 2,4 

WRITE (6,30) L1,L2,I,L1,L2,J,I,J 
1 CONTINUE 

30 FORMAT (1H , 4 (213 ,2X) ) 
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This would print out 

the 

indices of 

the 

instance, the output 

would look like 


11 

21 

12 

22 

11 

21 

13 

23 

11 

21 

14 

24 

11 

31 

12 

32 

11 

31 

13 

33 

11 

31 

14 

34 

11 

41 

12 

42 

11 

41 

13 

43 

11 

41 

14 

44 

11 

51 

12 

52 

11 

51 

13 

53 

11 

51 

14 

54 

Thus, the first row 

of V 

“u is 


>11 - u 

21 

U 12 + U 22 

= 0 

the second row is u 

11 

U 21 " U 13 + 

U 23 


so on. This is a linearly independent set 
rank(9 T ) =15 as it should. 

5.3.2 The Three-Way Classification Without 
action 

Suppose our model is 


1 s. For 


0 , and 
such that 

Inter- 



i = 1, • • • ,a; j - 1 } * * * ,b ; k = 1, • • • ,e; l = 1 , • • • , n 

1 J K 

subject to no two-way interactions and no three-way 
interactions. This means 


u . . 
ID * 

u. . . 

1 1 J * 

- u . . . + u . , . , 

id • I'd'* 

= 0 

(1) 

U * jk " 

u . . , 

* D k 

- U . . , + U . , , , 

• j k ' • ] ' k ' 

= 0 

(2) 

U i‘k " 

u . . . 

1 1 • k 

- u i-k’ + u i'-k' 

= 0 

(3) 

u. ... - 
1 D k 

“ u • i * i 

x 1 ] k 

U i j ' k + U i ' j ' k 



- 

U . + 

xjk 1 

Vjk' + U ij-k' 





U x ' j ' k ' 

- 0 

(4) 

all i. 

i', j. 

J’ 3 k, k« . 




To generate a linearly independent set satisfying 
( 1 ) , we use 


u 


ll' 


u _ - u -, T + u T , - 0 

il* 1J* IJ* 


and let I = 2 , • • • ,a , and J = 2 , • • • ,b in the same 
manner as in Section 5 • 3 • 1 • A set satisfying (2) can 
be obtained by using 


U -ll - U -J1 - U -1K + U -JK = ° 


and letting J - 2, • • • ,b; K = 2 , * • • ,c 
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For (3) we use 


u 


1*1 


U I • 1 u l 


K + U I * K 


0 


As I = 2,***,a and K = 2 , • • • ,c . For a set satis- 
fying (4) , we use 


u 


111 


u 


ill 


U 1J1 + U I Jl 


" U 11K + 


U I1K + 


U 


1JK 


U 


IJK 


for I = 2 , •••,&; J = 2,***,b; K = 2, • • • ,c . We must 
fix I and J and then let K vary. In computer 
notation, we have a nested "do-loop." 


For example, suppose 


I = 1, 2, 3; J = 1, 2, 3; K = 1, 2, 3 . 

The set representing no (I,J) interaction is 


u ll- 

- 

21* 

- U 12- 

+ U 22 • 

= 0 

u 

- u 

- u n ^ 

+ u ^ 

= 0 

11 • 

21* 

13 • 

23* 

• 

H 

H 

3 

- U 3.1 * 

U 1 2 • 

+ U 32- 

= 0 

u n* 

" U 31- 

- U 13- 

+ f J 3 3- 

= 0 


The sets representing no (J,K) and no (I,K) inter- 
actions can be obtained in an analogous manner. 
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representing no 

(X,J,K) 

interaction 

is 

u lll " 

U 2 2 1 ^ 

U 121 + 

U 2 21 



U 11 2 

+ U 212 

+ U 122 

" U 2 2 2 

* 

0 

U lll 

U 211 

U 1 21 + 

U 221 



- U 113 

U 213 

U 123 

U 223 

= 

0 

U lll - 

U 211 

U 131 + 

U 231 



- U 112 

+ U 212 

+ U 132 

U 2 3 2 

= 

0 

U lll " 

U 211 

U 131 + 

U 231 



U 113 

+ U 213 

+ U 13 3 

U 2 3 3 

= 

0 

U lll " 

U 311 " 

U 12 1 + 

U 3 21 



‘ U 112 

+ U 312 

+ U 122 

U 3 2 2 

= 

0 

U lll " 

U 3 11 

U 121 + 

U 321 



- U 113 

+ U 313 

+ U 123 

U 3 2 3 

= 

0 

U lll " 

U 311 ~ 

U 131 + 

U 3 3 1 



U 11 2 

+ U 31 2 

+ U 132 

U 332 

= 

0 

U lll - 

U 311 

U 131 + 

U 3 31 



- U 113 

+ U 313 

+ U 133 

U 3 3 3 

= 

0 
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is a (20x27) matrix of 


Thus, for this example 9 T 
rank 20 . 

5.3.3 Split Plot 

As shown in Chapter 3, we can consider this as 
a three-way classification with no (J,K) and no (I,J,K) 
interaction. Thus the model would be 


subject to 



u . + e . .. 

13 k 13 k 


• j 'k 


'• jk ' 


• . j ' k 1 


= 0 


U ijk " U i ' jk - U ij'k + U i -j'k 
u ij k ' + u i ' jk ’ + u ij 'k * 


- u. , . , .= 0 

1 ' j ' k ' 


T 

for all i, i * , j, j', k, k ’ . We construct 0 by 

finding the linearly independent set for no (J,K) 
interaction and the set for no (I, J,K) interaction. 
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5.3.4 Missing Cells 


The procedure for constructing 0 T when there 
are missing cells is as follows. 

(1) Construct 6 as though no cells were 
missing. 

(2) Eliminate the u..'s that do not exist in the 

13 

model . 


Consider this example; y 
i = 1,2,3; j = 1,2,3, and k 


u . . + e . , , 
13 13 


0 , 1 , 


,n. . 
13 


where 






X \ 

1 

2 

3 

1 

X 


X 

2 

X 

X 

X 

3 

X 

X 

X 


In other words, u 12 is not in our model. Suppose 
the constraints are no (I, J) interaction. Now we 
write 


u 


ll 


- u 


ll 


“ U U + 


u 


IJ 


0 
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and generate 


u il * u 2l * U 12 + u 22 “ 0 


U 11 u 21 " u !3 + u 23 " 0 


U 11 ~ U 31 U 12 + U 32 “ 0 


U 11 U 31 U 13 + U 33 0 


Now we subtract (3) from (1) and obtain 


U 31 “ U 32 U 21 + U 22 0 


Now 


U 31 ” u 2l U 32 + U 2 2 ° 


U 11 “ U 2 1 ^ U 13 + U 23 0 


U 11 " U 31 " U 13 + U 33 = ° 


( 1 ) 

( 2 ) 

(3) 

(V) 


are a set of linearly independent constraints which 
span the set of possible constraints. Hence we take 

T t 

8 u to be the above three equations. 
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5.3.5 Balanced Incomplete Block 


As shown In Chapter 3, we can consider this as 

a two-way classification without interaction and with 

missing cells. For example: y. . = u, . + e. . sub- 

1 3 13 

ject to no (I, J) interaction. Suppose we observed 


\j 

i Nv 

1 

2 

3 

1 

X 

X 


2 

X 


X 

3 


X 

i 

X 


that is u 31 » U 2 2 » a nd u !3 are not in the model. 


Again we write the constraints as though there 


were no missing cells, that is 


U 11 

U 21 

U 12 

+ 

U 22 

0 

(1) 

U, „ 

- u„. 

¥ 

+ 

U „ = 

0 

(2) 

11 

21 

13 


23 



U 11 

U 31 

“ U 1 2 

+ 

U 32 = 

0 

(3) 

u n 

" U 31 

" U 13 

+ 

U 3 3 

0 

(4) 


157 




We must eliminate u 31 , u 22> and u 13 from the above 
equations. We subtract (4) from (3) and obtain 


U 13 ~ U 12 U 33 + U 32 


Subtracting (2) from (1) we obtain 


- U 12 + U 13 ' U 23 + U 22 = 0 


Subtracting (6) from (5) we have 


— U 3 3 + U 32 + U 23 * U 22 = 0 


or 


u 


22 


- U 33 + U 3 2 + U 23 


Substituting u 22 into (1) we have 


U 11 U 21 “ U 12 + U 3 2 U 33 + U 23 


Thus 


(5) 


( 6 ) 


0 


e T u 


= u 


11 


- u 


21 


- u 


12 


+ u 


3 2 


- U 


33 


+ U 


23 


5.3.6 Lat in Square 

The Latin Square can be considered as a three- 
way classification with no interactions and with 
missing cells. 
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Suppose we observe 


Column 


Row 


1 2 3 


y lll 

y 122 

y i33 

y 212 

y 223 

y 2 31 

y 313 

y 3 21 

y 332 


Here we have a 3*3 Latin Square. We can consider 


the model as y. = u, + e. subject to no 

''13k 13 k 13 k 0 

Interactions where 1 = 1,2,3; j = 1,2,3; and 

k = 0,1,2, n. . . Let us first write 0 T if we had 
* .13 

T 

no missing cells. Now 0 can be constructed by 

using the results of Section 5.3.2. We would then 

eliminate all the u... *s except u..., u. 0 „, u. 

13k 111 122 13 3 7 


U 212» U 22 3* U 231» U 313» U 321» U 332 
this is 


The result of 


0 T u 


1 -1 0 0 1-1 
1 0 -1 -110 


-1 

0 


0 

-1 


1 

1 


u 


Naturally we would not use this approach if we had 
no missing cells since standard computing formulas are 
readily available. 
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5.4 Tests of Hypothesis 

Previously, we have assumed we had the hypo- 
thesis to be tested. There was no discussion about 
whether these hypotheses were appropriate. Our 
attitude has been that if an experimenter came to 
us with a model and wanted to test a given hypothesis, 
say H q : A u = 0 , then we provided the correct 

test. It was the experimenter ' s problem to decide 
what he wanted to test. This attitude seems to con- 
tradict most textbooks on applied statistics and 
most computer programs that analyze linear models. 

These textbooks and programs provide the reader 
or user with one set of hypotheses that are to be 
tested. They seem to say, "Here are the answers, 
we hope you have asked the questions that correspond 
to these answers." 

Let us take a simple example. Suppose an 
experimenter came to us with data taken in a com- 

I 

pletely randomized design in which he had three 
"treatments." He knew before he ran the experiment 
that the effects due to the "treatments" were not 
equal. What he wanted to know was, "Is the effect 
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of treatment X twice the effect of treatment 2 plus 
treatment 3?" 

Now he runs his experiment, obtains the data, 
and calls on the local statistician to analyze his 
data. The statistician is overjoyed at having such 
a simple textbook problem. In no time he gives the 
experimenter the usual computer output: 


Source of Variation 

df 

ss 

MS 

F 

"Treatments" 

a-1 

XX 

XX 

fine 

Error 

n-a 

XX : 

XX 



As the statistician hands the output to the experi- 
menter, he happily explains, "Your F value is 
highly significant." The experimenter, seeing the 
statistician is pleased about the significant F , 
also is pleased and leaves with great admiration for 
the mystic powers of the statistician and his science. 

Farfetched? Not at all; similar occurrences 
happen daily. The statistician is given data and 
asked to analyze it. He puts the data into one of 
the "can" programs, gets the standard output, and 
thinks he has done his job. This is fine as long as 
he is in the simple balanced case. But things change 


l6l 




when he tries to analyze data in the unbalanced case. 

In the first place, the literature on analyzing such 
data is quite confusing. This problem stems, possibly, 
from trying to tie the analysis of linear models to 
the analysis of "main” effects and interactions. 

For example, suppose we are given the following 
table : 


Source of Variation 


A 

a-1 

B 

i — 1 

I 

rQ 

Error 

(a-1) (b-1 ) 1 


It is customary to refer to A and B as main 
effects. There would be general agreement on the 
interpretation given to them. If we are given: 


Source of Variation 


A 

(a-1) 

B 

(b-1) 

AB 

(a-1) (b-1) 

Error 

ab (r-1 ) 


we still say A and B are main effects and AB is 
interaction. Now the trouble begins. A number of 


162 






texts indicate that "A" means that we are testing 
H q : a: = 0, i ■» l,***,a . They give this inter- 
pretation to both of the previously given tables. 
But then we read. 


"In the presence of interaction, it is 
rarely useful to ask about main effects 

of either factor Despite questionable 

meaning, mean squares for the main effects 
are commonly reported even when Interaction 
is present. This is a routine practice 
which is open to criticism, but is almost 
unavoidable . " 

One text defines the main effect A "as a measure of 
the change in the response variable to changes in the 
level of the factor averaged over all levels of all 
the other factors." But in the two-way model with 
interaction, this same text indicates the test for 
A implies testing H q : a 1 = • • • = a & • There is 
no averaging here. Also the fact that we are in a 
balanced case allows us to do things that are not 
valid in the unbalanced case. For instance, consider 
the following two tables: 


Source of 
Variation 

\if 

SS 

A 

a-1 

SSA 

B 

(b-l) 

SSB 

AB 

(a-1) (b-l) 

SSAB 

Error 

ab(n-l) 

E 
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and 


Source of 
Variation 

df 

SS 

A 

a-1 

SSA 

B 

b-1 

SSB 

Error 

abn-a-b+1 

E* 


Note that one difference in the two tables is that 
E * = E + SSAB and that SSA and SSB remain the same. 

If we have unbalanced data, this is not true. We 
would get an SSA’ and SSB’ as well. In the unbalanced 
case, what does SSA, SSB, SSA' and SSB* signify? 

Suppose we are in the "missing" cell case and 

the model is y.., = u + a. + 3. + 6. . + e. .. . One 

ijk i j x;jk 

textbook says "it is now necessary to assume that 
there are no empty cells, else the main effects are 
not estimable..." Yet another text presents methods 
to obtain the sum for "main" effects in this case. 

This brings us to another point. Are the 
restrictions placed on the parameters (or random 
variables) an integral part of the model or are 
they used simply to help solve the normal equations? 
Again, if we are in the balanced case, it really 
makes little difference. But what about the 
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unbalanced case! The use of Y\ a . - 0 rather 

■*— ' i 

than 0 can result in some very con- 

fusing answers in the analysis of the two-way 
interaction design. Also, if we are in the mixed 
model, the expected values of the mean squares depend 
upon whether we use the restrictions as an integral 
part of the model. 


Summing up, it seems that an analysis has been 
developed for certain models in the balanced case. 

This analysis was tied to the idea of main effects 
and interactions. When other models were developed, 
their analyses were forced to flow along lines 
similar to the balanced models. Since the techniques, 
concepts and interpretations were special to the 


balanced case, they could not be carried over directly 
to the unbalanced case. There is no need to tie the 
analysis to one form. In fact, there is no reason 


to think that experimenter A and experimenter B 
want the same analysis to 

I 


y ijk 


u + a . + v . + <5 . . + e . . , . 

i j ij xjk 


It is absurd to think both men are interested in the 
same hypotheses. Their analyses should depend upon 
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the reasons for which they conducted their experi- 
ments. Their experiments should have been designed 
to answer certain questions. 

Does this "u ,f -model approach free the statis- 
tician from all responsibility in formulating the 
hypothesis to be tested? Yes and no. The statis- 
tician, in his role as consultant, will have to take 
the objectives of his client and express them in 
meaningful mathematical terms. As we all know, the 
mere statement of the objectives is quite an accom- 
plishment. Also, in certain models, the statistician 
can present hypotheses suggested by the model. For 
example, suppose we had a 3 X 3 Latin Square with one 
missing observation. Let us suppose U 11;L is mis- 
sing. And suppose we would like to suggest some 
possible hypotheses to be tested. 


1 2 3 


A 

B 

G 


U 122 $ 

u , „ 
13 3 

B 

C 

A 

U 212 

U 223 

U 231 

C 

A 

B 

U 313 

U 3 21 

U 332 


u 


1 • • 


u. 


u. 


u 


• 1 * 


u 


2 * 


u 


» 3 • 
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Some possible hypotheses are 


( 1 ) 





u 


3 * • 


3 


( 2 ) 

(3) 


u 


u 


u 


H 


V Vi-. + W- + K 3 U 3. 

23^^^ is not necessarily zero, 


and so on. 


0 where 


Another example is the following two-way classi- 
fication with interaction 

12 3 


1 

u n 


U 13 

2 

U 21 

u o 
2 2 

U 23 

3 


U 32 

U 33 

4 

U 41 

U 4 2 



What are appropriate hypotheses for this model? The 
first, of course, should be to see if the model is 


additive. 

Suppose 

not , then what 

can we 

could test 

the following: 




H : 
o 

u li = 

U 21 

= U 41 


H : 
o 

U 2 2 

U 32 

U 42 


H : 
o 

U 13 

U 23 

U 3 3 
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Or we could use a test like the one suggested by 
Elston and Bush [12], Again, we should define the 
purpose of the experiment and ask questions with 
that in mind. 

In conclusion, let us note that throughout 
Chapter 3 we postponed the discussion of the "merits" 
of various hypotheses to this section. Yet nowhere in 
this section did we discuss the "merits" of these 
hypotheses. The reason for this is that, while any of 
the hypotheses that were posed are legitimate, their 
value or merit depends upon the purpose of the experi- 
ment. The researcher and the statistician have a 
joint responsibility to pose meaningful hypotheses 
that are to be tested. Finally it might be better if 
we did not tie hypotheses to the idea of "main" effects 
and interactions. 
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CHAPTER 6. NUMERICAL EXAMPLES 
6,0 Preliminaries 

We will now consider several numerical examples. 
These examples were run on the UNIVAC 1108 using a 
program written in FORTRAN V. The program calculates 

T * 

the sum of squares for H q : Xu = 0 in the following 

manner . 

(1) X T is orthogonalized, that is = H 1 X T 

where H 1 is chosen so that 

A^ = h 1 x t xh^ = I 

(2) X^ = H^X^ where H_ is chosen so that 
v 2 2 1 2 

\ T 2 A(X T X)~ 1 k T \ 2 = H 2 X^A(X T X)" 1 a' I 'x i H2 = D 

(3) ss(x T ) = S(x 2 ) = ^ g- U^) 2 

i=l i 

This procedure is explained in detail in Chapter 4 . 
Because of a lack of space, f this chapter does not 
consider all the intermediate steps needed to calcu- 
late the sum of squares. 
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6.1 The One-Way Classification 


Consider the following example taken from 
Ostle [24]. 


The model is 


1 

2 

3 

4 

5 

7.3 

5.4 

8.1 

7.9 

7.1 

8.3 

7.4 

6.4 

9.5 


7.6 

7.1 


10.0 


8.4 





8.3 





is 

y ij 

= u . 

X 

+ e . . 
ID 



where 


u . = u + a . 

i x 


The hypothesis to be tested is 


H : u. = u„ = u, = u. = u c 

o 1 2 3 t 4 5 


which is equivalent to H 


a = a = a = a =a 
1 2 3 4 5 


in the " 3" model. 
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Thus A^u = 0 is 


Let 


A' 


D = 


1 -1 

0 0 

0 




1 0 

-1 0 

0 

u 

0 


1 0 

0 -1 

0 




_1 0 

0 0 

- 1 _ 




8617 

-.3463 - 

.1258 - 

.3463 -. 

0432” 

1713 

.3691 - 

.8321 

.3690 -. 

0773 

0 

.7071 

0 

- 

.7071 

0 

1679 

.2095 

.3031 

.2095 ~. 

8900 

2382 

0 


0 

0 


0 

.4488 


0 

0 


0 

. 0 

.3333 

0 


0 

0 


0 

•8730_ 



Then H q : A u = 0 is equivalent to H q : 

where A T A = I and A T (X T X) _1 A = D . 


A*u = 0 


Now u = u* = (X T X) 1 X T Y . 
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T a 

But (XX) is diagonal, hence u is just the cell 

means, that is 


u = 


8.0 

6.6 

7.3 

9.1 

7.1 


and 


SS(A T ) 


5 "i ( x i 5 )" 


= 10.66 


and 


$ 2 


[Y - Xu] T [Y - Xu] 
_ 7 __ 


_ 7.17 _ 


,80 


and 


P = 


10.66 

3T7807 


= 3.34 


Now 


suppose <- N^Q,a 2 ^ , and we wish to 


estimate a using the above SS(A ). From Eq. 4.7, 

cL 

we see that % 


•2 _ SS(A T ) - 4q 


-2 


where k Q = ^ = 10.56 
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Hence 


and the V 


%2 _ 10.66 - 4( .80) 

a 10.56 

Is, from Eq. (4.8), 


.7064 


HO ■ ? • 


i 4 + i— <j 2 a 2 + i| + — a 4 

» k o a k 2 L 9 J 


where k, = 




d 2 

1 


= 32.898 or 


V 


(0 • 


•59a 4 + • 38a 2 a 2 + .104a 4 

a a 


which agrees with the results obtained from the for- 
mula in Searle's paper [29]. 


o 

Now suppose we wanted to estimate o & from 

H : A T u = 0 where 

o 

A t = i - [0 1 0 -1 0 ]' 

/2 

% 

Then we see that D = .3333 and 


SS(A T ) 


1 

.3333 


(X T u) 2 



J (6.25) 


= 9-375 
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Thus 


where 


or 


and 


* 2 - S.375 - 3.20 

a ~ k o 

k o = SdT = .3333 = 3 

a 2 = iz^Ll = 2.058 

a 3 




, 4 2 2 

+ 5 - O O 
3 a 


4 - 104 

+ ST“ ° 


2 a 4 + 1.33a 2 a 2 + 1.28a 
a a 


^2 

It is clear that a has a smaller variance 

a 

^2 

than . By the above procedure, we can generate a 

rather large set of unbiased estimates and find the 
variances of the estimates In a rather efficient 
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manner. This approach is contrasted with Searle's 
paper [29] in which it took considerable effort to 
develop an expression just for the variance of one 
unbiased estimate. 

6.2 The Two-Way Classification Without Interaction 
6.2.1 No Missing Cells 

Let us consider the example taken from Harvey [ 18 ] . 


Ration No. 

1 

Sire No. 2 
3 

The model is 


1 2 


5 

6 

2 

3 

2 5 7 

3 6 

8 9 
8 

3 

4 6 7 

4 6 


i = 1, • • • ,3; 
subject to 


j 


u 

u 


y . . , = u . . + e . . . 

id l D k 




= 1,2; k 

= 1,** 

• 3 n . . 

X3 

— U 

— u 

+ u ^ 

11 

21 

12 

22 

— U 

— u 

•f u 

11 

31 

12 

32 


0 

0 
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The matrix A (X T 'X) _1 A T is given in Section 4.2.2. Now 
(u*) T = (5.5, 2.5, 4.6, 8.3, 3.0, 5.4) 

(u) T = (3.19, 4.81, 5.39, 7.01, 3.65, 5.27) 

o 2 = 4.02 

Let us test H : u. = u . , and H : u . = u . . . 

The corresponding hypothesis matrices are also given 
in Section 4.2.2. 


Now 


and 



= 15.68 
* 9.707 



15.68 

r-^urm 


1.95 


F 


2 


9.707 

4.02 


2.41 


Now suppose that before the experiment was run we, 
thought that u 22 = 2 u i;l . Therefore, another test is 
H : u = 2u . Most texts do not treat hypotheses 
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like this; however, the sums of squares can be 
obtained by 


SS(A T ) = (A T u) T [A T A(X T X)“ 1 A T A]” 1 (A T u) 
where A T = [2 0 0 -1 0 0] 

or 

ss(aT> - £ f X I ") 2 * = 1 ' 1 ' 4 

i-1 1 ' ' 


and 



The estimates of variance components are discussed 
in Section 4.2.2. 
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Now 


a 2 = 8.183 

and 

(u*) T = (2.0, 4.6, 7.7, 2.3, 4.1, 5.5, 9.2, 

3.0, 8.55, 13.2, 3.37, 5.75, 11.0) 

(u) T = (1.84, 4.01, 8.30, 1.55, 3.82, 5.99, 

10.3, 3.74, 8.17, 12.47, 3.76, 5.73, 

10 . 22 ) . 

Now suppose we test H : u. . = u. . . and 

o x j I'd 

H : u.. = u... , then the appropriate sums of 

o id id' 

squares are, respectively, 

«(*;) = 
ss ( a i) - 

T 

Again we can test H q : X u = 


37.86 

215.23 

T 

0 for arbitrary A 
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6.2.3 BIB Design 


The following example is taken from Graybill [153. 


Total Weights of Nitrogen in Grams of Six Cuttings 
of Alfalfa Forage Grown in Greenhouse Pots 


Block j 

Treatments i Block total Y . 

j D 

1 

■ 

2.10 2.67 2.91 j ' 7-68 

( 1 ) ( 2 ) ( 4 ) | 

2 

1.14 3.00 3.10 j 7.24 

( 2 ) ( 3 ) ( 5 ) j 

3 

2.92 3 . 1 ^ 2.99 j 9-05 

( 3 ) W ( 6 ) j 

•-i 

4 

3.13 03 2.75 | 8.51 

( 4 ) ( 5 ) ( 7 ) 1 

5 

2.84 3.13 1.85 : 7-82 

( 5 ) ( 6 ) (10 j 

6 

3.01 2.99 1.82 i 7-82 

( 6 ) ( 7 ) ( 2 ) | 

i 

7 

2.86 OO 3.78 ; 10 . l 8 

( 7 ) ( 1 ) (3) 1 


k = 3,t = 7, b = 7, r = 3, X = 1 
Grand total Y ### = 58.30 


T 

The model is y = u + e i; . subject to 6 u = 0 
where 
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0 0 01 -1 0 1 -1 0 -1 1 00 0 0 0 00 0 0 0 
0 0 0 1 -1 0 0 0 0 -1 0 1 1 -1 0 0 0 0 0 0 0 

0 0 01 0 -1 0 00 -1 100 0 0 -1 01 0 0 0 
0 0 0 1 0 -1 0 0 0 -1 0 0 0 1 0 0 0 0 -1 1 0 

0 -1 1 0 0 0 0 0 0 0 0 0 0 -1 1 0 0 0 1 0 -1 

0 1 -1 0 0 0 -1 00 0001-0 -1 0 00 0 0 1 
0 1 -1 0 0 0 0 00 0000 0 0 0 -1 1 0 -1 1 
1 0 -1 0 0 0 0 00 -1 010 0 0 0 00 -1 0 1 


and where 


£ U H> u 15 5 u 17 > u 21 > U 22 s U 26* U 3 2 , 


U 33 J U 3 7 3 U 4 1 3 U 43* U 44» U 52 J U 54 5 


U 55 * U 6 3 3 U 65> U 6 6 3 U 74 , U 76 J U 77^ 
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Q > 


.2424 and 


2 


2.0999999 


_ 2. 4233332“ 

1.8500000 


2. 0704761 

3.5399999 


2.9961904 

2.6700000 


2. 0461904 

1.1400000 


1. 6290476 

1.8200000 


1.9547619 

3.0000000 


2.8404762 

2.9200000 


3.0290476 

3.7800000 


3 .8304762 

2.9100000 


3.2104761 

3.1400000 

vN 

u = 

2.9819047 

3.1300000 


2.9876190 

3.0999999 


2.7704761 

2.6300000 


2.9647619 

2.8399999 


2.8347618 

2.9899999 


3.0390476 

3.1300000 


2.9147619 

3.0100000 


3.1761904 

2.7500000 


2.5576191 

2.9899999 


2.6890476 

2.8600000 


3. 3 5 33 3 3 2_ 


suppose we 

test 

(i) 

H ; 
0 

u . . = u . . . 
3-D 3-D 

(2) 

H : 
0 

u . . = u . . , 
ID ID 
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(1) Treatment "adjusted" for blocks. 

(2) Blocks "adjusted" for treatments. 

(3) Treatments "unadjusted." 

(4) Blocks "unadjusted." 

In tabular form, we have 


I Source of 

1 Variation 

df 

SS 

H : 
o 

u . . = u . , , 

13 13 

6 

3.2708 

H : 
o 

u . . = u . . , 

13 13 

6 

1.5370 

H : 

o 

u. = u . , . 

6 

3.7885 

H : 
o 

u . = u . . 

• 3 * 3 

6 

2.0347 

1 Error 

8 

1.9388 


Now If we are in a fixed model and are interested in 

just H : u. . = u. , . , then we see that 

0 o xj x ' j 


P 


3.2908 

" 6 • (2424) 


2.263 • 




And if we are in a mixed model, where treatments are 
fixed and blocks are random, we see that 

E[SS(A t )3 = 6o 2 + 14 a 2 

D 

T 

where A is associated with H : u . , = u . . , 

O 1J 1J 


Now 


.2424 

1.5375 - 6( 


T W 


) _ 


.006 


Also 


and 


u . . - u . . . = a . - a . , 

ij i'j i x 1 


l(u. - u. . ) - a. - a. , 

X* 1 ' * X X ' 


T 


x T 

Let X, be such that X,u = u. , - u. . . and 


•3 - 


1 1 


X l be 


such that 


X^u = u. - u. , 

2 x • x 1 • 


Now X*u and X^u both 


are unbiased estimates of a.-a... Let o,. = V 

X X 1 11 


22 


' V ( X 2 S ) 


and a 12 = Cov 


( T' a * TaI 

XfU, X 2 u J , 


11 
Then 


(»F). 


rp /\ 

X^u 


"■or, , 

a. 



11 

12 

rp a 

Xlu 

— 

a. 

0 

2 


L 12 

22 J 


- V 
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Now from the discussion In Section 4.2.3, we 


know a better estimate of a. -a. would be a 


( x r) + a 2 ( x ?) 


where 


a. 


V~ 1 £ 

£ T V -1 £ 


and where = [1 1] . But 


r il* °12 


and a „ are 
2 2 


2 2 

functions of a and a 


b ’ 


However, if we use the 


/\ 2 a 2 *p /s 

a and a b as described above, then X 1 u and X 2 u 

^—1 T 

will be uncorrelated with V since A satisfies 


the conditions described in Section 4.2.3. Hence an 

+ a , 

1 X * i \ X J 

where 

r 

a. 


l( X F) + a 2 ( X 2^) 


a 2 

L 2 J 


A -1 
V 1 £ 

T A — 1 

rv 


For example, suppose we wanted to estimate a 1 “ a 2 * ^e 


could use Xu. = - u 21 


377 and 


T ^ 1 r ^ A a /s /v A 

X 2 U = 2 ^ U 15 + U 17 u 22 " u 26 + U 37 “ U 32 


+ U 55 U 52 + U 65 U 66 + U 77 U 76^ 


= 1.47 
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u + Ae where u 


Now u = 

E[u] = yl + u 1 a . 


= yl + U ; a + V^b and 


V[u] = 


ajv.vj + a 2 A 

b 1 1 


Now A^u and A^u are unbiased estimates of a 1 _a 2 


where 


where 



VlA^u] 


■ + d n° 2 


L n 


.86 


= ° 


Hence 


V 


(>?) • 


.36a‘ 


Also 


(■?) ■ 


■MyIs a !2 = 2 


'22 


A T V V A 
2 112 


3 

9 


And 
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Thus 


v(**G) = 3o 2 + 9 a 2 

Cov /aJG, = <Jb' > 'I V l V l*2 + 0 2 aJa(X t X)' 1 A T X 2 

= 0 . 

/\ 2 a 9 

Now substituting a and a b which we found 
above, we have 


X^u 


.860 2 

0 

W A 

-2 U _ 


0 

3o 2 + 9ol 


2085 0 

0 .7812 



1 

4.796“ 


“.789 

6.076 

_1.280_ 


211_ 
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Hence another unbiased estimate of a 1 -a 2 

a i( X l^) + a 2( X 2 a ) = - 297 + * 310 

= .607 

This is the same estimate as the one obtained by 
Graybill [15 ] , where he used the "interblock" and 
"intrablock" estimates. Let us exhibit another esti- 
mate obtained from the general procedure. Let 


•P /\ 

X *u 

A /N, 

= U X1 - u 

21 ■ 



m /\ 
*2 U 


A /S 

U 17 “ U 22 

- “26 + “37 

- 

3 2 


+ "55 - 

“ 5 2 + “65 - 

“66 + “77 

- “ 7 6 ] 


= 1.47 




X 3 U 

" \ [U 11 

+ u 15 + u 17 

- u„, + u„„ 
21 2 2 

' “26 


= .620 


We see that X^u , X 2 u and X»u are unbiased estimates 
of a 1 -a 2‘ ^ow 

v(x^u) = + d 33° 2 
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where 


33 


2 

3 


X 3 V * 1 V I X 3 


16 

9 


Therefore , 


V 


(>?) ■ 


16 2 2 2 

9~ °b + 3 a 


and 


Cov 


Cov 


Now 


( X I S > *?) 

(^2 fi » X 3“) 


xjAX 3 a 2 


2 rr 2 

3 ° 


X 2 V X X 3 0 b + X 2 AX 3 C ' 2 


| a 2 + 2a 2 
3 b 


V = 


2085 

0 

.1616 

0 

.7812 

.1618 

1616 

.1618 

.1726 


and V is uncorrelated with X:£u, X^ 11 an<a ^ 


W >3 



Hence 


A _ I 

v 2 ± 

T A — 1 

rv a 


.6226 

.1661 

>2113 


Thus, another unbiased estimate of ..a -a 

a i( x ! 2 ) + a 2 ( x 2“) + a 3( x 3 a ) * - 60 " 

Let us find an estimate of a 1 ~d 2 using the con- 
cepts developed in Section 4.2.4. There we saw that 
another unbiased estimate was X u where 


u = [i - (x T v" 1 x)" 1 e[0 T (x T v _ 1 x)" 1 0]" 1 e T ] 


• [x T v _ 1 x] 1 x T v 1 y 


where V is an estimate of the covariance matrix of 

Y . 


In our case, we have 


u = [I - V0(0 T V0)" 1 0 T ]Y 
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Now X T u = \x 11 - u 21 is an unbiased estimate of 
a l -a 2 and 1 u - -376 . In this case., V is almost 
diagonal since the nonzero off-diagonal terms are 
.006. Hence u and u are not significantly dif- 
ferent . 

6.3 The Two-Way Classification With Interaction 
6.3.1 No Missing Cells 

Let us consider the example given in Section 6.2.1. 
T 

Since 0 = 0 , we have that 

(u) T - (u*) T 

= (5.5, 2.5, 4.6, 8.3, 3.0, 5.4) . 

Now suppose we test 


(1) 

H : 
o 

u . = u . , 

i* i ' • 

(2) 

H : 
o 

u . = u . , 
*D •j' 

(3) 

H : 
o 

u.. - u . . . -u.., + u . . . . = 0 
id i'd id id 

Note that 

(3) 

is equivalent to H : 0 u = 0 where 


0 T is defined in Section 6.2.1. 
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The results are tabulated below 


Hypothesis 

df 

SS 

H : u = u . , 

2 

' 

21.00 

H : u , = u . , 

O VD *D' 

1 

3.59 

H : 6 T u = 0 

o 

2 

30.22 

Error 

12 

26.07 


To illustrate, let us find the expected value of 

SS (A ), where A is the matrix associated with (2) 

Now A T = — [1 -1 1 -1 1 -1] . 

/F 

Suppose we assume the following structure on u 

u = yl + U « + V x b + V 2 6 

where and U a denote the fixed effect, V^b and 

/ 2 ' 

V 6 denote the random effects, and where b ~ N 1 0 ,a I 

" / 2 \ ' ' 

6 ~ N f 0 ,c g I I . Let us note that some authors [28] 

assume that 6 ~ N(0 ,V) where V has some nonzero 


off-diagonal elements. 

Naturally,, their results will 

differ from ours. 

Now 

u = u + (X T X) ~ 1 X T e , hence 

E[u] = 

+ 
i — i 

U x o 

V[u] • 

a b V l V l + a 6 V 2 V 2 + a2 < xTx ) 1 
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Note that V = I , Hence 


Now 


Now 


v[u] = ff?v.v* + oh + 0 2 (x T x) _1 

Dll 0 


E[SS(A T )] = i- [[A T V(u)A] + (A T U) 2 3 

1 


u. 


~1 

0 

0 


~1 

o - 

1 

0 

0 


0 

1 

0 

1 

0 

V , = 

1 

0 

0 

1 

0 

1 

0 

1 

0 

0 

1 


1 

0 

_0 

0 

1 _ 


_0 

1_ 


Hence 


and 


Hence 


A T u = A T yl + A T U 1 a 


= 0 


A T V(u)A 


■ (*\ V J A )°Z 


+ a 2 A t AI + (A t (X t X) 

o 


0 2 . 2 , , 2 
3 a b + + d i a 


EtSSCA 1 )] - 


3 2,1 2 2 

-s 0, + J 0. + 0 

d^ b d^ o 


~ 1 K)o 



where 


d 1 = A T (X T X) 1 A 

41 

90 


Hence 

E[SS(A T )] = a 2 + 2.195a? + 6.585a? 

0 D 

2 

Note that there is a a g term in this equation. This 
does not agree with those authors who assume 
6 ~ N(0,V) . 

6.3.2 Missing Cells 

We shall consider the example given in 

r ip 

Section 6.2.2. Here 0 = 0 . The major question is 

which hypotheses are to be tested. Let us test the 
following . 


(1) 

H : 
o 

u . 

= u . , 

X 1 * 

(2) 

H : 
o 

u . 

9 3 

= u . « 
* D 

(3) 

H : 
o 

e T u 

= 0 


f 

4 ? 

where 0 T is defined in Section 6.2.2. Now 
(u) = (u*) 

= (2.0, 4.6, 7.7, 2.3, 4.1, 5.5, 9.2, 3.0, 

8.55, 13.2, 3.37, 5.75, 11.0) 
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The results are tabulated below 


| Hypothesis 

df 

SS 

1 H : u. - 

1 o x* 

u. . 

1 ' * 

3 

36.66 

w 

0 

• 

LJ. 

II 

u . , 

3 

201.95 

H : e T u = 

O 

0 

6 

7.7461 

Error 


13 

0.4366 


This concludes the examples on the two-way classifica- 
tion with interaction. 

6.4 Latin Square 

This example is taken from Snedecor [32]. 


Cows 

1 2 3 


1 

A: 

608 l B: 

OO 

GO 

VJ1 

■ MB* 

o 

940 

Period 2 

B: 

715 ! C: 

1087 ! A: 

■ 

776 

3 

C: 

844 : A: 

a 

a 

711 | B: 

m 

832 

The model is 





^ijk 

= "a . . , + 

ljk 

e . 

13 k 


i = 1,2,3; j = 1,: 

P *3 * 

- > P y 

k = 0 ,l,n . . 

subject to 

no 


interactions . 
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Hence 0 u is 


1-1 0 0 1 - 1-1 0 1 ! 

1 0 -1 -1 10 0 -1 "'ll 


where 


^- U lll U 122 U 133 U 212 U 223 U 2 3l 


U 313 U 321 U 332^ 


Now we test the following hypotheses 


(1) 

H : 

o 


li 

to 

• 

• 

= U 3 * * 

(2) 

H : 
o 

M.. 

= u. 2 . 

= ^3* 

(3) 

H ; 
o 


U • • 2 

II 

• 

• 

CJ 


Let us consider (3) in detail. 


U lll + U 321 + U 231 


U 122 + U 212 + U 332 


U 133 + U 223 + U 313 


In terms of the "g" model 


u. = u + a. + y . + 6, 

i}k x j k 
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Hence 


u 


• • 1 


Y-i + Y 9 + Y, a + a + a 
u + 6 1 + — | + — J ~ 


u 


Y, + Y, + Y. a + a + a 
u + J 2 + _i 1 3 + _1 2 3 


u 


Y 1 Y 2 y 3 a i a 2 a 3 

U + S 3 + I ^ + ~ 1 ^ 


Hence H: u ,=u „ = u 0 is equivalent to 

o ‘‘l • • 2 • • 3 ^ 

H: = 6^ = 6_ in the model. The results are 

0x23 

given below. 

(u*) T = (6o8 , 885, 940, 715, 1087, 776, 844, 

711, 832) 


( u) T = (711.3, 883.3, 759.7, 931.7, 886.7, 

696, 868, 823) 


1 Hypothesis 

df 

SS 

H : 

o 

U l« • U 2 • • 

= U 3*. 

2 

5900. 

H : 
o 

U * 1« = U • 2 • 

1 U * 3 • 

2 

47214 . 

H : 

u..i = u.. 2 

II 

• 

• 

OJ 

2 

103436. 


Error 


2 

4843. 
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Let us now consider the case where we have a missing 
cell. Suppose y = 6 08 , is missing. The model 
remains the same except that 


6 T u = [1-1-1011-1 0]u 


where 


T 

U = ^ U 1 2 2* U 133 J U 212 3 U 223 3 U 231 J U 313* 


U 321 5 U 3 32 ^ 


Again we have the problem as to what we should test. 


Let us test 

H : 
o 

A T u = 0 

or 



H : | 

U 212 

“ U 2 31 ~ 

U 313 

+ U 332 

= 0 

O 

U 223 

U 122 

U 231 

+ U 133 

= 0 


This is equivalent to H q : <$i = 6 2 = <$ 3 in the "3" 

model. The results are given in the, following, table . 


Hypothesis 

df 

ss 

H : A T u = 0 

o 

Error 

2 

1* 

80960 . 

2773.0 
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6.5 Simultaneous Confidence Intervals 

Let us consider the example in Section 6.1. 
That is y. . = u. + e . . where i = 1.2,3. 5 , 

xj x ij 1 55 5 3 

j = 1 , 2 , • • • ,n . Now suppose we wanted an a% con- 

fidence interval for A u. From Section 2.4, we see 
that the set of all K such that 


U - A T G) T (A T (X T X)~ 1 A)~ 1 (g - A T u) < p 

s £2 - (a ,s ,n-p) 


is an a.% confidence interval, where F, .is the 

u% point of a central F distribution with s and 
n-p degrees of freedom. Let us take a = .95 and 



1 0 

0 0 

0 

U = 

1 

H 

l ^ 


1 

O 

O 

0 1 

0 


L U 4 J 


Now the set of £ such that 


U - A T u) T (A T (X T X)" 1 A) 1 (? - A T u) „ ,, 

2TTM1 — — — < 4.2b 


is a 95$ confidence interval for 


u. 


u. 
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Or we have 
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CHAPTER 7. CONCLUSION 


7.0 Summary 

In the preceding chapters we have developed a 

comprehensive theory of linear models. We have shown 

that by formulating the model as y i j k - + e i j k » 

2 

where y . .. ~ N(u. ., a ) and where certain relations 
may be known about the u^.. , we could analyze this 
model in a very efficient manner. It was shown that 
the one theory can be applied to the balanced case, 
the unbalanced case, the missing cell case, the case 
of interactions and the case of no interactions. We 
also showed that the one theory can be applied to both 
classification models and design models. By using 
this approach, we are able to analyze mixed models, 
that is we can estimate components of variance and 
provide estimates of fixed effects based upon 
information recovered from the random effects. We 
also applied this approach to regression and 
covariance models. 
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There are a number of advantages to using the 
"u" model approach. First, let us contrast it 
with the usual approach used to teach the balanced 
case. Here the student is given a set of rules and 
a set of equations. He is told that with these he 
can analyze certain designs. For example, in the 
two way classification with interaction, he is told 
that the following table is the way to analyze such 
a model: 


Source of 
Variation 

df 

SS 

MS | 

A ' 

a-1 

A 

yy 

>! 

B 

b-1 

B 

yy 

B 

yy 

AB 

(a-1) (b-1) 

AB 

yy 

KB 

yy 

Error 

ab ( r-1 ) 

E 

yy 

E 

yy 


where 





y. 


) 


2 


■? 

4 


B 

yy 



y.. 


.) 


2 
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a b _ _ __ _ 

AB = n V V' (y • ■ - y . - y . + y ) 

yy Z~j 1 3 * 1 * • * 3 * 

1=1 j=l 


E = AB - A - B 
yy yy yy yy 


Some intuitive justification is given as to why 
the degrees of freedom and sums of squares arise as 
they do. Other tests of hypotheses are almost never 
considered. Also, some authors include certain 
restrictions as a part of their models. For example 


y ijk 


u+a. + 3 . + <$ . . + e . 

x 3 13 13k 


whe re 

IX = Eh ■ EX - Ex - 0 

• » • • 

1 J 1 J 

The table above answers only three questions; 

that is, it tests only three hypotheses. As we know, 
there are infinite hypotheses one can test. But the 
student is told to test the same hypotheses each time 
he conducts such an experiment. 
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This approach is especially harmful to those 
students from other disciplines that have had only 
two or three "methods" courses. In those courses, 
we taught them only the rules of statistics rather 
than the rules and concepts. As a result, they do 
not understand the true role of statistics or of 
the statistician. They think statistics is limited 
only to certain models that can be analyzed and only 
certain questions that can be answered. Using 
standard computer programs perpetuates this mis- 
conception. Persons laboring under this miscon- 
ception are the ones who give us boxes of data cards 
that we as statisticians are supposed to analyze by 
our "mystic" methods. More importantly, these 
students, from their experiences in the "methods" 
cpurses, have the mistaken idea that there are only 
four or five classical models that are valid to 
statistical analysis. They might make wrong 
assumptions about the data generated by their experi- 
ments in an effort to make those ..experiments fit one 
of the classical models. Or they might disregard 
some of the data in order that their experiments 



will fit the balanced design of the classical 
statistical analysis. The result could be that a 
model which does not describe the experiment is used. 
Thus, the "u" model provides a method of analysis 
that enables the experimenter to pose an essentially 
unlimited number of hypotheses. It also allows a 
great amount of flexibility in choosing the model. 

Second, the statistician is like a merchant — he 
is a merchant of models. When a scientist comes to a 
statistician, the scientist has conjectured that a 
phenomenon he has observed or will observe may fit a 
certain model. He asks the statistician to help 
formulate the model so that quantitative analysis can 
be performed. The statistician looks over his ’’stock” 
of models and tries to find one that approximates the 
model presented by the scientist. If the statistician 
can offer only balanced models, or if the scientist 
can only ask predetermined questions, their efforts 

degenerate into a game of data shuffling which has 

% 

little or no scientific value. 
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Statisticians should be experts In models. They 
should formulate abstract models, analyze them, and 
investigate their advantages and limitations so that 
when they are called upon to help the scientist, their 
"stock” is such that they can be of help. They will 
also be able to determine whether the questions posed 
can be answered by a given model. Thus the second 
advantage is that the "u" model offers a general 
abstract model that can be applied to a large number 
of practical problems. 

Finally, there are several approaches to 
analyzing linear models. One is the generalized 
inverse. Another is the "u" model approach outlined 
in this paper. The claim here is not that the "u" 
model is the only approach, rather that it is one 
method that is conceptually simple. It is based upon 
the most fundamental statistical assumptions, .The 
student can be shown how to pose the questions, how 
to test them, and how to interpret them without having 
to understand the mathematics involved. Yet the 
principles they learn will apply to all types of 
linear models and not just to balanced models. 
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7.1 Future Research 


We will now discuss several research topics 
which arose from studying the "u" model. 

7.1.1 Other Models 

Presently, the "stock" of models is primarily 

linear models. We should seek to include nonlinear 

models. Also, the tests of hypotheses are restricted 

to the form H : A T u = E, . We should generalize this 

o 

T 

to include hypotheses of the form H q : A u £ B, , 

H : A T < C , etc. 

7.1.2 Vari an ce Comp onents 

Perhaps the use of the "u" model can reveal 
something about the properties of the estimates of 
the components of variances in the mixed models. 

7.1.3 Estimates of Fixed Effects in the Mixed Model 

We have seen that we need not restrict ourselves 
to the BIB design or to the classical interblock and 
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intrablock estimates in estimating fixed effects. 
However, the criteria we used in finding estimates 
was that the estimates must be unbiased. We should 
consider other criteria, such as minimum variance and 
so on. 
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